Category: linux

linux 没有真正的线程, linux 上的线程是轻量级进程LWP.

The main difference between a light weight process (LWP) and a normal process is that LWPs share the same address space and other resources like open files etc. As some resources are shared so these processes are considered to be light weight as compared to other normal processes and hence the name light weight processes.

获取进程的线程情况

➜  ~ ps -Lf 1210
UID        PID  PPID   LWP  C NLWP STIME TTY      STAT   TIME CMD
mysql     1210     1  1210  0   10 4月17 ?       Ssl    0:02 /usr/sbin/mariadbd
mysql     1210     1  1478  0   10 4月17 ?       Ssl    3:24 /usr/sbin/mariadbd
mysql     1210     1  1479  0   10 4月17 ?       Ssl    0:01 /usr/sbin/mariadbd
mysql     1210     1  1480  0   10 4月17 ?       Ssl    0:00 /usr/sbin/mariadbd
mysql     1210     1  1481  0   10 4月17 ?       Ssl    0:00 /usr/sbin/mariadbd
mysql     1210     1  1492  0   10 4月17 ?       Ssl    0:00 /usr/sbin/mariadbd
mysql     1210     1  1566  0   10 4月17 ?       Ssl    0:00 /usr/sbin/mariadbd
mysql     1210     1  9933  0   10 5月16 ?       Ssl    0:00 /usr/sbin/mariadbd
mysql     1210     1  9937  0   10 5月16 ?       Ssl    0:00 /usr/sbin/mariadbd
mysql     1210     1 13703  0   10 00:11 ?        Ssl    0:00 /usr/sbin/mariadbd

➜  ~ ls -l /proc/1210/task/
总用量 0
dr-xr-xr-x 7 mysql mysql 0 5月  16 13:44 1210
dr-xr-xr-x 7 mysql mysql 0 5月  17 00:12 13703
dr-xr-xr-x 7 mysql mysql 0 5月  16 13:44 1478
dr-xr-xr-x 7 mysql mysql 0 5月  16 13:44 1479
dr-xr-xr-x 7 mysql mysql 0 5月  16 13:44 1480
dr-xr-xr-x 7 mysql mysql 0 5月  16 13:44 1481
dr-xr-xr-x 7 mysql mysql 0 5月  16 13:44 1492
dr-xr-xr-x 7 mysql mysql 0 5月  16 13:44 1566
dr-xr-xr-x 7 mysql mysql 0 5月  17 00:12 9933
dr-xr-xr-x 7 mysql mysql 0 5月  17 00:12 9937

➜  ~ top -Hp 1210
top - 00:18:19 up 29 days,  7:20,  1 user,  load average: 0.21, 0.15, 0.13
Threads:  10 total,   0 running,  10 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  1881996 total,    85684 free,   900112 used,   896200 buff/cache
KiB Swap:        0 total,        0 free,        0 used.   791844 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
 1210 mysql     20   0 1158504 130800   7732 S  0.0  7.0   0:02.17 mariadbd
 1478 mysql     20   0 1158504 130800   7732 S  0.0  7.0   3:24.05 mariadbd
 1479 mysql     20   0 1158504 130800   7732 S  0.0  7.0   0:01.59 mariadbd
 1480 mysql     20   0 1158504 130800   7732 S  0.0  7.0   0:00.00 mariadbd
 1481 mysql     20   0 1158504 130800   7732 S  0.0  7.0   0:00.05 mariadbd
 1492 mysql     20   0 1158504 130800   7732 S  0.0  7.0   0:00.00 mariadbd
 1566 mysql     20   0 1158504 130800   7732 S  0.0  7.0   0:00.00 mariadbd
 9933 mysql     20   0 1158504 130800   7732 S  0.0  7.0   0:00.22 mariadbd
 9937 mysql     20   0 1158504 130800   7732 S  0.0  7.0   0:00.24 mariadbd
13703 mysql     20   0 1158504 130800   7732 S  0.0  7.0   0:00.00 mariadbd

➜  ~ pstree -p 1210
mariadbd(1210)─┬─{mariadbd}(1478)
               ├─{mariadbd}(1479)
               ├─{mariadbd}(1480)
               ├─{mariadbd}(1481)
               ├─{mariadbd}(1492)
               ├─{mariadbd}(1566)
               ├─{mariadbd}(13703)
               ├─{mariadbd}(15228)
               ├─{mariadbd}(15229)
               ├─{mariadbd}(15230)
               └─{mariadbd}(15231)

ref

what-is-the-difference-between-lightweight-process-and-thread

经常使用 top 查看监控系统进程情况，但是里面很多指标还是不够清晰。
* 为什么 VIRT 有时候会比系统内存还大
* RES 比 配置jvm 的Xmx还大
* top 中那些指标是关于内存的

通过 top 命令

CODEandDATA需要按F,然后使用空格键选中，才会显示出来

top -p 1210

PID USER      PR  NI    VIRT    RES    SHR   CODE    DATA   SWAP S %CPU %MEM     TIME+ COMMAND
1210 mysql     20   0 1158504 130804   7736  22472 1052816      0 S  0.0  7.0   8:53.50 mariadbd

VIRT

36. VIRT  --  Virtual Memory Size (KiB)
     The total amount of virtual memory used by the task.  It includes all code, data and shared libraries plus pages that have been swapped out and pages that have  been mapped but not used.

VIRT=CODE+DATA+shared libraries +pages that have been swapped out+pages that have been mapped but not used

SWAP

27. SWAP  --  Swapped Size (KiB)
        The non-resident portion of a task's address space.

被 swap-out 的内存页大小

RES

17. RES  --  Resident Memory Size (KiB)
    The non-swapped physical memory a task is using.

一个任务正在使用的，没有被swap-out 的物理内存

CODE

 4. CODE  --  Code Size (KiB)
        The amount of physical memory devoted to executable code, also known as the Text Resident Set size or TRS.

可执行代码驻留的物理内存总量，驻存代码集合（Text Resident Set, TRS）

DATA

6. DATA  --  Data + Stack Size (KiB)
     The amount of physical memory devoted to other than executable code, also known as the Data Resident Set size or DRS.

data 是 VM_WRITE & ~VM_SHARED & ~VM_STACK 与 VM_STACK 占用内存页之和，也就是所有非栈内存中可写但非共享内存页与栈内存页之和

$ANON = RES – SHR$ ( ANON 表示在堆上分配的内存)

$ANON <= DATA$ (vm_physic)

SHR

21. SHR  --  Shared Memory Size (KiB)
        The amount of shared memory available to a task, not all of which is typically resident.  It simply reflects memory that could be potentially shared with other  processes.

        任务可用的共享内存量，但并非所有的共享内存都是常驻（resident）的。它（SHR）只是反映了可能与其他进程共享的内存

SHR contains all virtual memory that could be shared with other processes, and RSS contains all memory physically in RAM that is used by the process.

Thus all shared memory currently in RAM is counted both in SHR and in RSS, so SHR + RSS has no meaning since it can contain duplicates counts.(SHR + RSS没有意义，因为他们可能包含重复的项)

1. 除了自身进程的共享内存，也包括其他进程的共享内存
2. 虽然进程只使用了几个共享库的函数，但它包含了整个共享库的大小
3. 计算某个进程所占的物理内存大小公式：RES – SHR
4. swap out后，它将会降下来

PR

16. PR  --  Priority
    The scheduling priority of the task.  If you see `rt` in this field, it means the task is running under real time scheduling priority.

    Under linux, real time priority is somewhat misleading since traditionally the operating itself was not preemptible.  And while the 2.6 kernel  can  be  made  mostly preemptible, it is not always so.
    在linux下，实时优先级有些误导，因为传统上操作本身不是可抢占的。虽然2.6内核可以使其成为可抢占的，但并不总是如此。

实时优先级取值范围是[1,99]，它对应的就是0-99的优先级，优先级较高。

In most cases PR value can be computed by the following formula: PR = 20 + NI.

Theoretically the kernel can change PR value (but not NI) by itself. For example it may reduce the priority of a process if it consumes too much CPU, or it may increase the priority of a process if that process had no chance to run for a long time because of other higher priority processes. In these cases the PR value will be changed by kernel and NI will remain the same, thus the formula “PR = 20 + NI” will not be correct.

NI

11. NI  --  Nice Value
    The nice value of the task.  A negative nice value means higher priority, whereas a positive nice value means lower priority（值越小优先级越高）.  Zero in this field simply means priority will not be adjusted in determining a task's dispatch-ability.
    任务的静态调度优先级，取值范围是[-20,19]。

Linux实际上实现了140个优先级范围，取值范围是从0~139，这个值越小，优先级越高。nice值的-20到19，映射到实际的优先级范围是100-139。

通过 proc filesystem

cat /proc/1210/statm
289626 32701 1934 5618 0 263204 0

//os 内存页大小

getconf PAGESIZE
4096

Table 1-3: Contents of the statm files (as of 2.6.8-rc3)

Field	Content		与 top 相关字段
size	total program size (pages)	(same as VmSize in status)	$$VIRT=289626*4096/1024=1158504$$
resident	size of memory portions (pages)	(same as VmRSS in status)	$$RES=32701*4096/1024 = 130804$$
shared	number of pages that are shared	(i.e. backed by a file, same as RssFile+RssShmem in status)	$$SHR=1934*4096/1024=7736$$
trs	number of pages that are ‘code’	(not including libs; broken, includes data segment)	$$CODE=5618*4096/1024=22472$$
lrs	number of pages of library	(always 0 on 2.6)
drs	number of pages of data/stack	(including libs; broken, includes library text)	$$DATA=263204*4096/1024=1052816$$
dt	number of dirty pages	(always 0 on 2.6)

pmap

➜  ~ pmap -X 1210|head -n 5
1210:   /usr/sbin/mariadbd
         Address Perm   Offset Device Inode    Size    Rss    Pss Referenced Anonymous Swap Locked Mapping
    556335aee000 r-xp 00000000  fd:01 22182   22472   5228   5228       5172         0    0      0 mariadbd
    5563372df000 r--p 015f1000  fd:01 22182    1392   1392   1392       1392      1392    0      0 mariadbd
    55633743b000 rw-p 0174d000  fd:01 22182     720    416    416        416       384    0      0 mariadbd
➜  ~ pmap -X 1210|tail -n 5
    7ffc7142e000 rw-p 00000000  00:00     0     132     76     76         76        76    0      0 [stack]
    7ffc714af000 r-xp 00000000  00:00     0       8      4      0          4         0    0      0 [vdso]
ffffffffff600000 r-xp 00000000  00:00     0       4      0      0          0         0    0      0 [vsyscall]
                                            ======= ====== ====== ========== ========= ==== ======
                                            1158508 131308 128832     131136    123272    0      0 KB

In computing, proportional set size (PSS) is the portion of main memory (RAM) occupied by a process and is composed by the private memory of that process plus the proportion of shared memory with one or more other processes(由该进程的私有内存加上与一个或多个其他进程的共享内存的比例组成). Unshared memory including the proportion of shared memory is reported as the PSS.

REF

top命令显示信息之谜

Linux top 命令里的内存相关字段（VIRT, RES, SHR, CODE, DATA）

The /proc Filesystem

Proportional set size

Writing to this will cause the kernel to drop clean caches, as well as
reclaimable slab objects like dentries and inodes. Once dropped, their
memory becomes free.

To free pagecache:
回收页缓存

echo 1 > /proc/sys/vm/drop_caches

To free reclaimable slab objects (includes dentries and inodes):
释放可回收的 slab 对象(包括目录和 inode)

echo 2 > /proc/sys/vm/drop_caches

To free slab objects and pagecache:

echo 3 > /proc/sys/vm/drop_caches

[root@node2 ~]# free -m
              total        used        free      shared  buff/cache   available
Mem:          48025       27055       12938         838        8031       19708
Swap:         32767         280       32487
[root@node2 ~]# echo "3" > /proc/sys/vm/drop_caches
[root@node2 ~]# free -m
              total        used        free      shared  buff/cache   available
Mem:          48025       27052       20032         838         939       19778
Swap:         32767         280       32487

来源: setting-proc-sys-vm-drop-caches-to-clear-cache)