计算机技术学习记录

Category: linux

  • shell中使用sed正则修改文件

    • sed 使用正则匹配
    • 在shell中 当变量有空格的时候使用双引号包围 “${var}”
    function change_version_class() {
    VERSION_CODE_FILE=$1
    BUILD_VERSION=$2

    if [ -f "${VERSION_CODE_FILE}" ]; then
        echo 'change '"${VERSION_CODE_FILE}"' info'

        BUILD_TIME=$(date +%Y%m%d-%H:%M:%S)
        BUILD_USER=$(whoami)
        BUILD_OS=$(uname -snr)
        BUILD_BRANCH=$(git branch |grep '[*]'|awk '{print $2}')
        BUILD_COMMIT=$(git log --oneline |head -n 1|awk '{print $1}')

        sed  -i 's/String BUILD_VERSION =\(.\)\+/String BUILD_VERSION = "'"${BUILD_VERSION}"'";/g'  "${VERSION_CODE_FILE}"
        sed  -i 's/String BUILD_TIME =\(.\)\+/String BUILD_TIME = "'"${BUILD_TIME}"'";/g'  "${VERSION_CODE_FILE}"
        sed  -i 's/String BUILD_USER =\(.\)\+/String BUILD_USER = "'"${BUILD_USER}"'";/g'  "${VERSION_CODE_FILE}"
        sed  -i 's/String BUILD_OS =\(.\)\+/String BUILD_OS = "'"${BUILD_OS}"'";/g'  "${VERSION_CODE_FILE}"
        sed  -i 's/String BUILD_BRANCH =\(.\)\+/String BUILD_BRANCH = "'"${BUILD_BRANCH}"'";/g'  "${VERSION_CODE_FILE}"
        sed  -i 's/String BUILD_COMMIT =\(.\)\+/String BUILD_COMMIT = "'"${BUILD_COMMIT}"'";/g'  "${VERSION_CODE_FILE}"
    fi
}

  • virsh 虚拟机xml信息,留个记录

    virsh dumpxml domain
    虚拟机dumpxml例,可以参考格式

    xml
    <domain type='kvm' id='12'>
    <name>ceph-node3</name>
    <uuid>4f52f828-f538-45cb-9f02-eeb555e96020</uuid>
    <memory unit='KiB'>8388608</memory>
    <currentMemory unit='KiB'>8388608</currentMemory>
    <vcpu placement='static'>2</vcpu>
    <resource>
    <partition>/machine</partition>
    </resource>
    <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.0.0'>hvm</type>
    <boot dev='hd'/>
    </os>
    <features>
    <acpi/>
    <apic/>
    </features>
    <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Haswell-IBRS</model>
    <feature policy='disable' name='hle'/>
    <feature policy='disable' name='rtm'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='xsaveopt'/>
    </cpu>
    <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    </clock>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
    </pm>
    <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
    <driver name='qemu' type='qcow2'/>
    <source file='/ceph2/images/ceph_node3.qcow2'/>
    <backingStore/>
    <target dev='vda' bus='virtio'/>
    <alias name='virtio-disk0'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
    <driver name='qemu' type='qcow2' cache='none'/>
    <source file='/ceph2/images/ceph_node3_storage.qcow2'/>
    <backingStore/>
    <target dev='vdb' bus='virtio'/>
    <alias name='virtio-disk1'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
    <driver name='qemu'/>
    <target dev='hda' bus='ide'/>
    <readonly/>
    <alias name='ide0-0-0'/>
    <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
    <alias name='usb'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
    <alias name='usb'/>
    <master startport='0'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
    <alias name='usb'/>
    <master startport='2'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
    <alias name='usb'/>
    <master startport='4'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
    <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
    <alias name='ide'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
    <alias name='virtio-serial0'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <interface type='bridge'>
    <mac address='52:54:00:94:b4:fe'/>
    <source bridge='br0'/>
    <target dev='vnet2'/>
    <model type='virtio'/>
    <alias name='net0'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
    <source path='/dev/pts/5'/>
    <target type='isa-serial' port='0'>
    <model name='isa-serial'/>
    </target>
    <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/5'>
    <source path='/dev/pts/5'/>
    <target type='serial' port='0'/>
    <alias name='serial0'/>
    </console>
    <channel type='unix'>
    <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-12-ceph-node3/org.qemu.guest_agent.0'/>
    <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
    <alias name='channel0'/>
    <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
    <alias name='input0'/>
    <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
    <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
    <alias name='input2'/>
    </input>
    <graphics type='vnc' port='5902' autoport='yes' listen='0.0.0.0'>
    <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
    <model type='cirrus' vram='16384' heads='1' primary='yes'/>
    <alias name='video0'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
    <alias name='balloon0'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </memballoon>
    <rng model='virtio'>
    <backend model='random'>/dev/urandom</backend>
    <alias name='rng0'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </rng>
    </devices>
    <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c21,c647</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c21,c647</imagelabel>
    </seclabel>
    <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+107:+107</label>
    <imagelabel>+107:+107</imagelabel>
    </seclabel>
    </domain>

  • 结合vmstat分析iostat输出结果

    man vmstat

    FIELD DESCRIPTION FOR VM MODE

    Procs

    • r: The number of processes waiting for run time.

      r 表示运行队列 (就是说多少个进程真的分配到CPU). 当这个值超过了CPU数目, 就会出现CPU瓶颈.

    • b: The number of processes in uninterruptible sleep.

    Memory

    • swpd: the amount of virtual memory used.
    • free: the amount of idle memory.
    • buff: the amount of memory used as buffers.
    • cache: the amount of memory used as cache.
    • inact: the amount of inactive memory. (-a option)
    • active: the amount of active memory. (-a option)

    Swap

    • si: Amount of memory swapped in from disk (/s).
    • so: Amount of memory swapped to disk (/s).

    IO

    • bi: Blocks received from a block device (blocks/s).
    • bo: Blocks sent to a block device (blocks/s).

    System

    • in: The number of interrupts per second, including the clock.
    • cs: The number of context switches per second.

    CPU

    • These are percentages of total CPU time.
    • us: Time spent running non-kernel code. (user time, including nice time)
    • sy: Time spent running kernel code. (system time)
    • id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.

    • wa: Time spent waiting for IO. Prior to Linux 2.5.41, included in idle.

      wa 列显示了IO等待所占用的CPU时间的百分比。这里wa的参考值为30%，如果wa超过30%，说明IO等待严重，这可能是磁盘大量随机访问造成的，也可能磁盘或者磁盘访问控制器的带宽瓶颈造成的(主要是块操作)

    • st: Time stolen from a virtual machine. Prior to Linux 2.6.11, unknown.

  • 使用iostat分析问题

    文章来源: How I Use Iostat and Vmstat for Performance Analysis

    考虑到svctm在iostat是不保证准确的. 分析io await相关问题需要和blktrace结合起来分析.

    1. Is the I/O heavy?

    • Check the sum of w/s and r/s. The larger, the heavier I/O.

    • Also check %util, the more, the heavier.

      • If it is close to 100, then the I/O is definitely significant. It should be noted that during writing,
      • if the disk is the bottleneck (%util is 100% for a long time), but the applications keep writing, as long as the dirty pages exceeds 30% of memory, the system will block all the write system call, no matter sync or async, and focuses on writing to the disk. Once this occurs, the entire system is slow as hell.
      • check wa in vmstat log.
        (more…)

  • n2n内穿透

    之前一直使用frp.然后看到frp 有个xtcp.试了一下,家里和公司完全穿透不起来.网上一查,n2n的效果很一些.这里是软件地址ntop n2n

    mac os

    1. 需要先安装tuntap
    brew update && brew upgrade brew-cask && brew cleanup && brew cask cleanup
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew cask install  tuntap
    1. 编译安装.由于新版本mac os系统限制.我把Makefile里安装帮助文档和工具部分注释掉了.
    install: edge supernode edge.8.gz supernode.1.gz n2n.7.gz
    echo "MANDIR=$(MANDIR)"
    $(MKDIR) $(SBINDIR) $(MAN1DIR) $(MAN7DIR) $(MAN8DIR)
    $(INSTALL_PROG) supernode $(SBINDIR)/
    $(INSTALL_PROG) edge $(SBINDIR)/
    #$(INSTALL_DOC) edge.8.gz $(MAN8DIR)/
    #$(INSTALL_DOC) supernode.1.gz $(MAN1DIR)/
    #$(INSTALL_DOC) n2n.7.gz $(MAN7DIR)/
    #$(MAKE) -C tools install

    ./autogen.sh
./configure
make

# optionally install
make install
    1. 启动
    sudo /usr/local/sbin/edge  proxy/config/edge.conf -f

    window

    1. 需要安装Visual Studio.我装的最新的Visual Studio 2019社区版.把C++和Windows SDK都安装上了.
    2. OpenSSL win32
    3. 在Visual Studio调整了编译为winX86.

    centos

    1. 最方便.直接看教程安装就行了.安装相关的库之后,执行:
    yum erase zeromq3 (Do this once to make sure zeromq3 is not installed)
yum clean all
yum update
yum install pfring-dkms n2disk nprobe ntopng cento
    1. 作为服务启动.
     systemctl start supernode
 systemctl start edge