|
|
 | | |  | The jail() function under FreeBSD |  |
par Yann Berthier (11/05/2001)
The jail(8) function under FreeBSD, part 2.
(yes, I know, there is no part 1, and I'm not sure there will be one either ;-)
In fact, parts 1 and 2 of the jail tip were merged into what we will call now
the part 2 - the actual tip)
0x00 Plan
0x01 Introduction - or, what is jail
0x02 A brief overview of jail
0x03 How to setup a jail
0x04 More applications in the jail
0x05 The future of jail
0x06 Conclusion
0x01 - What is jail
The traditional unix security model is simple, yet very effective: there is
the root account, and the others ;-)
Several projects are trying to add more granularity to this model : support of
Posix 1.e ACLs, and notions derivated from what is called 'trusted operating
systems', such as MAC and DAC access (to name just a few).
See http://www.trustedbsd.org/ for an implementation for the FreeBSD project.
All of those technologies have one shortcoming, though: to dramatically
increase the administrative costs.
The jail implementation under FreeBSD is a -very effective in my opinion- way
to enhance the granularity of privileges while retaining the Unix security
model and its efficiency :)
Jail was initiated by Poul-Henning Kamp of the FreeBSD project. Many credits to
him for his wonderful work !
His original presentation on the subject is available here:
http://www.nluug.nl/events/sane2000/papers/kamp.pdf
The jail() function appeared for the first time in FreeBSD 4.0
0x02 - A brief overview of jail
. It's a virtual FreeBSD environment bound to an IP address. There can be as
many jails as you set IP aliases
. Each jail is a fully functional environment, with it's own account database
and configuration files.
. A process in a jail is bound to its jail, and can't see processes outside
its jail, nor interfere with.
. You can delegate the root account of each jail while being able to enforce a
system wide policy. And if one service in a jail is compromised, the
attacker won't be able to escape the jail to leverage his gain to the rest
of the system - even if he gains root access in the jail via a local
exploit.
Let's take a look at an example:
You are an ISP doing multi hosting.
You want to let your customer administrate the services he offers (update of
his web pages, creation of CGIs, creation of accounts for his POP3 server, and
so on).
But you want to be sure your customer's settings can't affect your other
customers, and that, if his web site or whatever is compromised through a badly
written CGI script, or a new vulnerability is discovered in his FTP daemon, the
attacker won't be able to affect other customers, nor compromise the hosting
machine.
If you are in this case, jail is for you.
If you are not in this case, well, many chances are that jail is for you anyway
:)
0x03 - How to set up a basic jail
Convention: to distinguish between the guest environment (the jail) and the
host OS, the prompts of the commands contain the hostname:
ogoun for the host OS, whose main IP address is 10.0.0.75, and cell for the
jail.
# we create an IP alias on the ep0 network card ...
ifconfig ep0 alias 192.168.10.2 netmask 0xffffff00
# the directory of our jails will be /jail/$IP_ALIAS ...
mkdir -p /jail/192.168.10.2
# the show begins ... (we need the source tree of course)
cd /usr/src
export JAIL=/jail/192.168.10.2
make world DESTDIR=$JAIL
cd etc
make distribution DESTDIR=$JAIL NO_MAKEDEV=yes
cd $JAIL/dev
sh MAKEDEV jail
ln -sf null /boot/kernel/kernel
# to shut up the jail at boot time ...
touch ../etc/fstab
We use ipnat (from the ipfilter packet filtering tool) to redirect incoming
requests from the outside to the jail, and from the jail to the outside:
ogoun% cat /etc/filters/ipnat.rules
#
map ep0 192.168.10.2/32 -> 0.0.0.0/32 portmap tcp/udp 10000:65000
map ep0 192.168.10.2/32 -> 0.0.0.0/32
# redirecting the request to the ftp port of our host to the ftp
# port inside the jail
rdr ep0 10.0.0.75/32 port 21 -> 192.168.10.2 port 21
ogoun% ipnat -CF -f /etc/filters/ipnat.rules
To launch our jail :
ogoun% jail /jail/192.168.10.2/ cell.hsc.fr 192.168.10.2 /bin/tcsh
The jail seen from the inside ...
cell% ifconfig ep0
ep0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255
ether 00:50:04:cc:3e:cc
media: 10baseT/UTP
supported media: 10base2/BNC 10baseT/UTP 10base5/AUI
A line is added in the /etc/hosts file:
192.168.10.2 cell cell.hsc.fr
The /etc/resolv.conf file contains the IP address of our DNS server:
cell% cat /etc/resolv.conf
search hsc.fr
nameserver 1.2.3.4
/etc/inetd.conf is edited, and the only service left uncommented is ftpd.
Remember, we are talking about the /etc/inetd.conf file of the jail !
If we launch inetd:
cell% inetd
cell% netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 192.168.10.2.21 *.* LISTEN
Populating the jail with users - well, one user in our case ;-)
The user monkeytest is created with the adduser(8) facility:
cell% cat /etc/passwd | grep monkeytest
monkeytest:*:1001:1001:monkeytest:/home/monkeytest:/bin/tcsh
cell% cat /etc/group | grep monkeytest
monkeytest:*:1001:
We can't see processes outside the jail ...
cell% ps aux
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 48035 0.0 0.9 1488 290 p9 DWJ 2:17PM 0:00.14 /bin/csh
root 48921 0.0 0.7 1160 205 ?? DWsJ 2:17PM 0:00.01 inetd
root 8689 0.0 0.6 1196 200 p9 DWJ 2:31PM 0:00.01 su - monkeyte
monkeytest 8954 0.0 0.9 1428 268 p9 DWJ 2:31PM 0:00.03 -su (tcsh)
monkeytest 4108 0.0 0.3 576 78 p9 RW+J 3:29PM 0:00.00 ps aux
The flag 'J' in the STAT column let us know we are in a jail
In fact, many many other things let us know we are in a jail.
Keep this in mind if you think as a jail for a good honeypot basis.
However, a jail has several interesting properties as a honeypot:
. the outgoing flow can be filtered and
. the filtering rules can't be changed by the attacker,
. the jail can't be used as a platform to attack other networks.
. the administrator of the host OS can 'observe' the attacker without being
seen
. with a network sniffer (tcpdump)
. with a tty sniffer (watch)
. the log files in the jail can be looked at and kept safe outside of the jail
(with no possibility for the attacker to tamper with)
. the capability of FreeBSD to run native Linux binary is a plus, as most
exploits found today are targeted toward the Linux platform
(no troll intended, it's only because of the vast number of Linux machines
installed 'as is' straight on the Internet)
No need to say, one should not rely only on ipfilter of the host box, in the
event of a vulnerability of the jail code which would permit to escape it !
Back to our jail ...:
We have access to the outside world:
cell% ftp ftp.freebsd.org
Connected to ftp.freebsd.org.
220 usw3.freebsd.org FTP server (Version DG-4.1.73 983302105) ready.
Name (ftp.freebsd.org:yann): anonymous
331 Guest login ok, send your email address as password.
Password:
230 Guest login ok, access restrictions apply.
Remote system type is UNIX.
ftp> quit
221 Goodbye!
Except for raw sockets ...
cell% ping 1.2.3.4
ping: socket: Operation not permitted
And from outside the jail ...
ogoun% ifconfig ep0
ep0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
inet 10.0.0.75 netmask 0xffffff00 broadcast 192.70.106.255
inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255
ether 00:50:04:cc:3e:cc
media: 10baseT/UTP
supported media: 10base2/BNC 10baseT/UTP 10base5/AUI
ogoun% ps aux | grep J
root 48035 0.0 0.9 1488 290 p9 DWJ 2:17PM 0:00.14 /bin/csh
root 48921 0.0 0.7 1160 205 ?? DWsJ 2:17PM 0:00.01 inetd
root 8689 0.0 0.6 1196 200 p9 DWJ 2:31PM 0:00.01 su - monkeytest
titi 8954 0.0 0.9 1428 268 p9 DW+J 2:31PM 0:00.03 -su (tcsh)
yann 1617 0.0 0.5 1164 145 pa DW+ 3:29PM 0:00.01 grep J
The process 8954 is seen belonging to the user 'titi'. In fact, it's because
this user has the same uid (1001) as the user monkeytest inside the jail.
ogoun% netstat -an | fgrep '.21 '
tcp4 0 0 192.168.10.2.21 *.* LISTEN
One can now log via ftp to the ftpd server of the jail:
bubble:~$ ftp ogoun.hsc.fr
Connected to 10.0.0.75
220 cell.hsc.fr FTP server (Version 6.00LS) ready.
Name (10.0.0.75:yb): monkeytest
331 Password required for monkeytest.
Password:
230 User monkeytest logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
150 Opening ASCII mode data connection for '/bin/ls'.
total 193
drwx------ 2 monkeytest monkeytest 512 May 6 16:19 .ssh
-rw-r--r-- 1 monkeytest monkeytest 180935 Apr 19 20:32 thttpd-2.21b.tgz
226 Transfer complete.
ftp> quit
221 Goodbye.
ogoun% ipnat -l
List of active MAP/Redirect filters:
map ep0 192.168.10.2/32 -> 0.0.0.0/32 portmap tcp/udp 10000:65000
map ep0 192.168.10.2/32 -> 0.0.0.0/32
rdr ep0 10.0.0.75/32 port 21 -> 192.168.10.2 port 21 tcp
List of active sessions:
MAP 192.168.10.2 1178 <- -> 192.70.107.75 10001 [1.2.3.4 53]
RDR 192.168.10.2 21 <- -> 192.70.107.75 21 [192.70.106.33 2821]
How to launch the jail at boot time
The $JAIL/etc/rc.conf file should be edited to fit the needs of the jail.
cell% cat /etc/rc.conf
hostname="cell.hsc.fr"
syslogd_enable="YES" # Run syslog daemon (or NO).
syslogd_flags="-s -s"
inetd_enable="YES" # Run the network daemon dispatcher (or NO).
inetd_flags="-wW -a 192.168.10.2"
sendmail_outbound_enable="YES" # Dequeue stuck mail (YES/NO).
sendmail_outbound_flags="-q30m" # Flags to sendmail (outbound only)
sshd_enable="YES"
For the sshd daemon in the jail to be able to bind to the IP address of the
jail, you should say to your sshd daemon outside the jail _not_ to bind to this
IP address:
ogoun% cat /etc/ssh/sshd_config | grep -i listenaddress
ListenAddress 10.0.0.75
ListenAddress ::
And then ...
ogoun% cat /etc/rc.local
/usr/sbin/jail /jail/192.168.10.2/ cell.hsc.fr \
192.168.10.2 /bin/sh /etc/rc &
0x04 - More applications in the jail
An httpd server is compiled inside the jail (thttpd, see
http://www.acme.com/software/thttpd/).
The file /etc/rc.local is modified to launch the http server at boot time:
cell% cat /etc/rc.local
/usr/local/sbin/thttpd
A NAT entry should be added to redirect incoming connections to the port 80/tcp
of the host OS to the port 80/tcp of the jail:
ogoun% cat /etc/filters/ipnat.rules
#
map ep0 192.168.10.2/32 -> 0.0.0.0/32 portmap tcp/udp 10000:65000
map ep0 192.168.10.2/32 -> 0.0.0.0/32
# redirecting the request to the ftp port of our host to the ftp
# port inside the jail
rdr ep0 10.0.0.75/32 port 21 -> 192.168.10.2 port 21
# redirecting requests for the sshd server
rdr ep0 10.0.0.75/32 port 22 -> 192.168.10.2 port 22
# redirecting requests for the httpd server
rdr ep0 10.0.0.75/32 port 80 -> 192.168.10.2 port 80
Our jail now begins to be a fully functional environment ...
cell% ps aux
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 92416 0.0 0.3 580 92 pg RW+J 3:04PM 0:00.00 ps aux
root 33099 0.0 0.5 1080 165 ?? DWsJ 2:55PM 0:00.01 cron
root 17139 0.0 0.5 1044 152 ?? DWsJ 2:56PM 0:00.01 syslogd -s -s
root 28713 0.0 0.6 1160 190 ?? DWsJ 2:56PM 0:00.00 inetd -wW -a
root 30405 0.0 0.6 1080 178 ?? DWsJ 2:56PM 0:00.01 cron
root 32211 0.0 1.3 2312 394 ?? DWsJ 2:56PM 0:00.14 /usr/sbin/ssh
nobody 47140 0.0 0.9 1540 283 ?? DWsJ 2:56PM 0:00.01 /usr/local/sb
root 80634 0.0 1.0 2312 299 ?? DWsJ 6:07PM 0:00.71 sshd
root 87693 0.0 1.5 2404 481 ?? DWJ 3:04PM 0:00.15 sshd: monkeyt
monkeytest 89426 0.0 0.8 1428 259 pg DWsJ 3:04PM 0:00.05 -tcsh (tcsh)
root 90233 0.0 0.7 1200 215 pg DWJ 3:04PM 0:00.01 su -
root 90434 0.0 0.9 1448 276 pg DWJ 3:04PM 0:00.04 -su (csh)
cell% netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 192.168.10.2.22 *.* LISTEN
tcp4 0 0 192.168.10.2.80 *.* LISTEN
tcp4 0 0 192.168.10.2.21 *.* LISTEN
Active UNIX domain sockets
Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr
c93d6440 dgram 0 0 0 c933c180 0 0
c933c180 dgram 0 0 c9558ac0 0 c93d6440 0 /var/run/log
c93d61c0 dgram 0 0 0 c933c080 0 0
c933c080 dgram 0 0 c94b1900 0 c93d61c0 0 /var/run/log
0x05 - The future of jail: the near -current implementation ;-)
For now, several sysctl exist to control the behavior of jails:
ogoun% sysctl -a | grep jail
jail.set_hostname_allowed: 0
jail.socket_unixiproute_only: 1
jail.sysvipc_allowed: 0
Anyway, this is somewhat limited.
Here is a mail from Robert Watson in a FreeBSD mailing list regarding the
future of jail ...
This weekend I was spending some time tweaking the jail(8) code to improve
it's SMPng-happiness as well as manageability. Unfortunately, I ended up
rewriting it in the process :-). I changed the model somewhat so that
jails are now persistently configred, joined, et al, and broke out the
chroot() from the creation/joining process, as with increased namespaces
(such as System V IPC) creating a nice clean failure was increasingly
difficult. Aspects of individual jails may now be managed using sysctl's,
which appears to work reasonably well. Clearly there's a lot of work left
to do, but I'd appreciate comments if people are interested:
http://www.watson.org/~robert/jailng/
Simple example:
dev# ./jailctl
usage:
jailctl create [jailname]
jailctl destroy [jailname]
jailctl join [jailname] [-c chrootpath] [path] [cmd] [args...]
dev# ./jailctl create test
dev# sysctl -a | grep jail
jail.instance.test.sysvipc_permitted: 0
jail.instance.test.set_hostname_permitted: 1
jail.instance.test.socket_ipv4_permitted: 1
jail.instance.test.socket_unix_permitted: 1
jail.instance.test.socket_route_permitted: 1
jail.instance.test.socket_other_permitted: 0
jail.instance.test.ipv4addr: 0
dev# ./jailctl join test -c /tmp /bin/sh
# ps ax
PID TT STAT TIME COMMAND
907 d0 DWJ 0:00.02 /bin/sh
908 d0 RW+J 0:00.00 ps ax
# exit
dev# ./jailctl destroy test
dev#
I also have a jailinit(8) in the works which would allow improved
startup/shutdown in the style of init(8) (sans the whole sigchild thing).
Another feature I'd like to add is a jail signal call that allows a signal
to be delivered to all processes inside a jail from outside, allowing an
easier forceable shutdown.
0x06 - Conclusion
As it's clear with the mail from Robert Watson, jail is a work in progress, and
that many improvements will appear in a near future, notably regarding the
manageability of jails.
Anyway, jails are for now ready for production use, and are in fact used in the
real life. Give it a try, it's a powerful tool to compartment your FreeBSD
server.
To test a working example and see a jail in action, see the OpenRoot project:
http://sektor7.ath.cx/openroot/ where the initiator of the project give to
everyone the root access.
|
