BSDStore
management and repository for FreeBSD jails

work with cbsd


Attention! I apologize, but it is automatic machine translation of the text. You can improve it if will send to me more correct version of the text or fix html pages via GITHUB repository.

Jail operations

jail create

Commands:
 

% cbsd jconstruct-tui


% cbsd jconstruct


% cbsd jcreate jconf=/path/to/conf.jconf
Description:

The jail is created according to configuration file created by cbsd jcreate jconf=path_to_cfg. For create configuration, use a command cbsd jconstruct (question-answer dialog mode), or dialog(1)-based interface: cbsd jconstruct-tui, or via WEB interface. When pkg.conf and repository configured properly, you can preset to new jail some packages. For safety reason recommendet to use the official FreeBSD pkg repo or build you own package repository. If you wish to use pkg repo, from which the jail of cbsd created, place into your /usr/local/etc/pkg.conf strings: 

      PACKAGESITE:      http://dl.bsdstore.ru/freebsd/amd64/9.2/nox
(where 9.2 is your base version) and run: pkg update -f in shell

Please note: when repo unavailable, item pkglist in cbsd jconstruct-tui did not show anything.
Please note: multi-repo and and the ability to choose a specific repository from the list at the moment is missing, and will be used repository prescribed in pkg.conf of master host.

for create config via DIALOG:


% cbsd jconstruct-tui

If jconstruct-tui work correctly, on the exit the question for jail create will be asked. In a case positively answer, jcreate it will be executed on a new configuration automatically. Otherwise, the script will save configuration file (in $workdir/ftmp), on which it is possible to create a jail by hand.

Example (fill for jname, fqdn, ips fields + GO):


% cbsd jconstruct-tui




When repository configured properly, you can preset to new jail some packages. Use first character of name port for faster searching and hit space for mark it.




Required fields: a short (one word) the name of the jail. This name is used to identify the jail:


Required fields full (the domain) name of the cell. This name will be used as jail hostname:



Required fields: IP address of jail. May have a network prefix with a record like IP/prefix. To assign more than one IP address, use the editing of the file rc.conf jails described in Jail config



When you choose item GO, you are prompted to create the jail immediately by automatically running cbsd jcreate jconf=... Otherwise, you will be the path to the saved configuration to run cbsd jcreate by hand:

jail config

Commands:
% cbsd jconfig jname=jail1
Descriptions: configuration parameters jail

Each cell has its own rc.conf file in $workdir/jails-rcconf/, own the fstab file for a list of mounted file systems in the directory$workdir/jails-fstab/ and its own directory to store the statistics, descriptions of jails, configuration wizards and other supporting files in a directory $workdir/jails-system/

In the TUI-dialog can be brought, not all possible configuration options jails, in which case they can be corrected in the corresponding files via any text editor when jail is stopped

Jails IP address

IP addresses that are bound to the jail sets in $workdir/jails-rcconf/rc.conf_name file in the parameter ip4_addr. As an IP may serve as IPv4, and the IPv6 address. When starting and stopping jail, working with IP may take place in two modes - automatic on-the-fly creation of IP addresses for the jail at the time of launch + automatic removal from the interface IP when stopping or the use of previously initialized IP addresses.

When for jails assigned to more than one address, they should be listed separated by commas without spaces. IP can include network prefix specified through IP/prefix. By default, aliases created with the prefix /32, that may not be appropriate if the jail uses a separate subnet from the network server - in this case, the correct /prefix is needed.

The parameter that controls this behavior stored in $workdir/jails-rcconf/rc.conf_name file and called interface. If it is not in rc.conf-file or the value is 0, jstart and jstop will not be called ifconfig alias and ifconfig -alias, respectively. If its value is 1, before jail start, this command will be executed:

ifconfig alias interface ips,
and when jail stop: 
ifconfig inteface ips -alias
Be careful with this option, if you have only one IP for server that is used and this IP is assigned to the same jail: when stopping jail, ip address of the server will be removed automatically that will make the server unavailable. In this case, you need to use in jail rc.conf: interface=0 or just remove "interface" parameters from jail rc.conf. An example of a piece of configuration to create three IP addresses at the start of the jail: 
..
interface=1
ip4_addr="176.9.147.18/29,2a01:4f8:160:3002::1/64,192.168.0.2/24"
..

Mounting File Systems in jail

Each jail has own fstab file, which lists the file systems that are mounted in a jail at startup. This file is located in the file $workdir/jails-fstab/fstab.name and coincides with the syntax of the file system format /etc/fstab with the exception that, as the mount point indicated path relative to the jail root, not the master system. For example, if you want to jail1, which is located at
/usr/jails/jails/jail1
on the master host, mount tmpfs for /tmp jail dir (those actually in /usr/jails/jails/jail1/tmp from master filesystem), then the entry in the $workdir/jails-fstab/fstab.jail1 should look like this: 
..
tmpfs /tmp tmpfs rw 0 0
..

//to be continued

starting and stoping jail

Commands:
% cbsd jstart jname=jail1 jail2
% cbsd jstart jail1 jail2 ... jailX

% cbsd jstop jname=jail1 jail2
% cbsd jstop jail1 jail2 ... jailX

% cbsd jrestart
Description:

Running jails occurs at startup cbsd/server automatically if parameter astart in the corresponding jail file $workdir/jails-rcconf/rc.conf_name set to 1. When you stop the server or service cbsd, all running jails will be stopped automatically.

Starting the jail manually by the command:

 cbsd jstart jname=jail1
or 
 cbsd jstart jail1
or 
 cbsd jstart jail1 jail2 jail2 ..
(to launch multiple jails by one command)

When parallel=0 is set in $workdir/nc.inventory and you try to launch/stoping a few jails, start/stop will be held consecutively. It is not always good, as any error triggered inside jails in the rc-scripts, which leads to a pause, can block start/stop next jails.

In the case when parallel have a non-zero value, each next jails will be launched/stopped in N seconds after the previous one, where N - parallel value. When on this timeout previous jails does not have time to run/stop completely, the next jails will be executed in parallel.


To stoping jail jstop must be used, with the same syntax and behavior:

 cbsd jstop jname=jail1
or 
 cbsd jstop jail1
or 
 cbsd jstop jail1 jail2 jail2 ..
(to stopping multiple jails by one command)

With a large number of jails (particularly databases, with services such as MySQL, redis, cassandra, etc.), it should be borne in mind that the low value parallel (for example, less than 5 seconds) can generate a very intensive storage I/O load that can increase the amount of start-up time of all jails, than if they were run sequentially or higher timeout.

Additionally, when a shutdown command is running on the server with lots of jails/services that should be taken into consideration low timeout that defaults to perform rc.shutdown sequence. In this regard, the process init(8) can interrupt the rc-scripts on this timeout, resulting in abnormal shutdown jails. In this case, the database can lead to nonconservation or damage data. To avoid this, /etc/rc.conf master system should be adjusted parameter rcshutdown_timeout to a more reasonable value (default: 90 seconds)

In the absence of rcshutdown_timeout in the system /etc/rc.conf, cbsd initenv will put this option in its sole discretion automatically.

Also, keep in mind that when using the zfs features ($workdir/nc.inventory, the parameter zfsfeat=1), and the file system ZFS, at jstop, file system of jails will be unmounted and those catalog $workdir/jails-data/jail1-data will be empty. If in such a case when jail data is requires without running it, by the command zfs list You can see the name appropriate file system and run zfs mount fs.

jail remove

Commands:
% cbsd jremove jail1 jail2 ..
Description: Removal of jail mentions all files anyway connected with a jail:

a) config file rc.conf and fstab for current jail
b) data directory or ZFS filesystem with jail data
c) statistics and description for jail
d) snapshots

in a case when jremove executed for jail in status On, it will be automatically stopped.

Example:

% cbsd jremove jail1

jail renaming

Commands:
% cbsd jrename
Description: Performs renaming jail and relevant data in the new name

Can be executed only on the stopped jail.

As mandatory parameters:
old - old name of the jail
new - new name of the jail

As optional parameters:
host_hostname - FQDN, new full hostname of jail
ip4_addr - new IP address of the jail (if multiple IPs, separated by a comma with no spaces)

Example (renaming jail jail1 into jail50 with a FQDN and ip addresses change):

% cbsd jrename old=jail1 new=jail50 host_hostname=jail50.my.domain ip4_addr=192.168.0.5/24




working with NAT

Commands:
% cbsd natcfg
% cbsd naton
% cbsd natoff
Description:

jails do not always require external IP, or, for security reasons, a number of services need to deploy on private IPs, so they were not available from the Internet. Thus, the jails may be needed for Internet access.

In this case the NAT translating the private IP address of the jails to external IP of the server. cbsd functional has a configuration template NAT rules for translating of private networks RFC1918.

To do this, this command as the first step is required:
cbsd natcfg
for selecting the appropriate framework for which the configuration will be loaded NAT: pf, ipfw and ipnat.

Attention! When you configure this, system file /boot/loader.conf nodes will be modified to load the appropriate modules.

Choice framework also stored in the file $workdir/nc.inventory as a record:

nat_enable="name_of_framework"

IP, which will be used as source address, is requested when you first start cbsd initenv and stored in a file $workdir/nc.inventory, in the form of natip="IP", where it can be changed later.

To modified natip take effect, you must run cbsd natcfg and cbsd naton again.

Currently, the cbsd configuration NAT limited to the creation of rules for translating private networks. If you need to get something more from simple NAT rule, you can edit the rules file created manually in the directory $workdir/etc/ in files:

pfnat.conf, when PF is used
ipfw.conf, when IPFW is used, or
ipnat.conf, wnen using IPNAT from IPFilter

Note:
If nodeip (IP of nodes), he is within RFC1918 networks for the subnet broadcast NAT rule will not be created.

To disable nat control by cbsd, use the follow command:

cbsd natoff

jail list

Commands:
% cbsd jls
Description:

Show the list of jails on a local node or for all added nodes. At a conclusion by the local node, fields are shown: 
JNAME JID     IP             Hostname          Path      Status


JNAME - jail name
JID - Jail ID
IP - list of assigned IP addresses (IPv4,IPv6)
Hostname - FQDN jail
Path - root filesystem for jail

Status - On (running), Off (stoped), Sl (Slave mode, unregister)

note: jail in Slave status can not be started until it not switch to Off status (register) With Slave status, rc.conf, fstab and data directory are named as rc.conf.slave, fstab.slave and jname-data.slave. In a slave condition can there are replication for jail, backup. To switch the Slave/Master modes it is possible through cbsd jswmode command.

In a case when to the local server are added a key of remote of nodes, it is possible to receive the list of all jails in a farm via 
cbsd jls alljails=1
or 
cbsd jls alljails=1 shownode=1
for output with node name where jail are hosted.
In the output from cbsd jls alljails, it is possible to see only active jails (in status On)

Example:
% cbsd jls



jail login

Commands:
% cbsd jlogin
Description:

Execute login into jail from root user. In case when attempt in a jail which isn't present on a local node, but it is present on one of remote nodes, jlogin will ask a question on attempt to login in remote jail through ssh.

Example:
% cbsd jlogin kde4





work with jail parameters

Commands:
% cbsd jget

% cbsd jset
Description:

Example:

jail cloning

Commands:
% cbsd jclone

% cbsd jrclone
Description:

Carries out jail cloning in the new. As obligatory arguments, the jail source/original is specified throughold, new name via new arguments and FQDN (hostname) via fqdn. As the unessential - new ip of the address via ip4_addr (if several IPs, addresses separated by commas with no spaces)

Example (cloning jail amp123 to amp123clone with ip and fqdn changes):
% cbsd jclone old=amp123 new=amp123clone fqdn=amp123clone.my.domain ip4_addr=10.0.0.200/24




jail snapshots (zfs-only)

Commands:
% cbsd jsnapshot
Description:

Show the list, create, delete and rollback snapshots for a jail when node startend on ZFS filesystem and zfsfeat in file $workdir/nc.inventory set to 1. through mode parameter you can you specify that it is necessary to make. Possible options:

list - show snapshot liss for current jail
create - create snapshot for jail
destroy - delete snapshot for jail
destroyall - remove all snapshots for jail
clone - clone snapshot into new jail
rollback - rollback jail to current snapshot state

Additional arguments:

jname - for what jail action is made
snapname - snapshot name
snapfs -

Example:

create snapshot named gromozeka for jail1 jail:
% cbsd jsnapshot mode=create jname=jail1 snapname=gromozeka
create snapshot named zelepuka for jail1 jail: 
% cbsd jsnapshot mode=create jname=jail1 snapname=zelepuka

Run jail1 and stop after some modification: 
% cbsd jstart jail1
..
% cbsd jexec jname=jail1 cp /bin/date /root
% cbsd jexec jname=jail1 file -s /root/date
/root/date: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 9.0 (900506), stripped
% cbsd jstop jail1
..
Rollback jail1 to snapshot zelepuka state: 
% cbsd jsnapshot mode=rollback snapname=zelepuka jname=jail1
% cbsd jstart jail1
...
% cbsd jexec jname=jail1 file -s /root/date
/root/date: ERROR: cannot open `/root/date' (No such file or directory)



jail export

Commands:
% cbsd jexport
Description:

Attention: command execution allows on jail in status On. However it is necessary to remember (especially for jail with databases) when you import such jails, with a high probability it is possible to got problems with inconsistency filesystem in jails, old .pid files that can break work of the imported jails

Export jail into file (*.img). In jname arguments you can set jail for export. img-file stored in $workdir/export directory. Original jail after exports is not modified

Example (export mysqljail jail to mysqljail.img):
% cbsd jexport jname=mysqljail



jail import

Commands:
% cbsd jimport
Description:

Import jail from (*.img) file. In jname argument you can set fill path to file or just filename (without .img extension) if file placed in $workdir/import directory. Img-file after import is not modified.

Exampl (import jail from /tmp/mysqljail.img file):
% cbsd jimport jname=/tmp/mysqljail.img



backup and file replication for jail

404 Not Found

nginx

jail description

Commands:
% cbsd jdescr
Descriptions:

Each jail can have the description/summary. cbsd jdescr without arguments just out description for all jails on the local node.
Command
% cbsd jdescr mode=update jname=jname
start editor vi for entering description. By argument editor you can start alternative favorite editor (for example, who can UTF8) - vim, mcedit. Descriptions stored in ASCII-file, placed to $workdir/jails-system/$jname/descr file, where jname - name of jail. This file you can just copy. As directory jails-system participates in jcoldmigrate, jclone, jimport/export operations, description will be will remain.

Example (run mcedit editor for description edit for kde4 jail): 
% cbsd jdescr jname=kde4 editor=mcedit mode=update

jail cold migration

Commands:


% cbsd jcoldmigrate


Description:



cbsd jcoldmigrate does cold (with stoping) migration jail from one node to another.
Argmument node point for destination node. Preliminary, RSA/DSA key for remote node must be added via
 cbsd node mode=add operation. Also, on a remote node service rsyncd should work (cbsdrsync).



By default, the jail status on a new node is inherited - if the jail worked, it also will be 
automatically started on a new node. If the jail didn't work - remains in Off status.
To operate, what status should be on a remote jail not is dependent on a condition on the original, it is possible
through parameter start.



Jail on the source, after successful jcoldmigrate will be stoped and switch to Slave status (unregister). 
In rc.conf file of jail on the destination node there will be a record where this jail came from



Notes: For jcoldmigrate, the next action is performed (in process they pass automatically and aren't visible)
  • copy configuration files to remote node, status of jail set to Slave on remote node (cbsd j2prepare)
  • exec rsync, which does a full copy of the directory with data to remote node (cbsd j2slave)
  • stop jails (if running) on the source node (cbsd j2slave)
  • one more rsync job, for synchronization of those files which be modified
  • switch jail status on source to Slave (cbsd jswmode)
  • switch jail status to master on the remote node (cbsd rexe + jswmode)
  • If jail was running - start jail on remode node (cbsd rexe + cbsd jstart)

    Example (make cold migration for jail amp123 to netsnap node):
% cbsd jcoldmigrate node=netsnap jname=amp123
on the destionation node anything isn't present now:
from node cbuilder64 migrate jail amp123 to netsnap:
jail amp123 on netsnap was started automatically:

jail limits control

Commands:


% cbsd jrctl
% cbsd jrctl-tui


Description:



To view the statistics, set and unset limits on the jail, use the command cbsd jrctl.

This command with the keys:

% cbsd jrctl mode=apply  ...


              
% cbsd jrctl mode=unset  ...


              

automatically called for the install or removal of limits when working jstart or jstop respectively.

By command


              
% cbsd jrctl mode=show


              

you can see current statistics on the jail resources consumed, which can be used to generate reports and graphs
for loading jail, as well as the cbsd daemon used to generate recommendations on the need to add new resources and for overload warnings.

By jrctl you can set the following limits jail:

a) All you can do a framework FreeBSD rctl(8):


              
cputime            CPU time, in seconds
datasize           data size, in bytes
stacksize          stack size, in bytes
coredumpsize       core dump size, in bytes
memoryuse          resident set size, in bytes
memorylocked       locked memory, in bytes
maxproc            number of processes
openfiles          file descriptor table size
vmemoryuse         address space limit, in bytes
pseudoterminals    number of PTYs
swapuse            swap usage, in bytes
nthr               number of threads
msgqqueued         number of queued SysV messages
msgqsize           SysV message queue size, in bytes
nmsgq              number of SysV message queues
nsem               number of SysV semaphores
nsemop             number of SysV semaphores modified in a single semop(2) call
nshm               number of SysV shared memory segments
shmsize            SysV shared memory size, in bytes
wallclock          wallclock time, in seconds


              



b) Set the priority level of one over the other jail  renice(8), for example,
you may want to give for jail with distcc lowest priority, while the jail with HTTP server - the middle priority, and the jail with the database - is high priority.



c) In the case of file system ZFS, set a quota on the use of the file system.


// WIP





Example:




              
% 


              


              

                
              

              





              
Dynamic DNS for jail

              
Commands:



              
% cbsd ddns


              

Description:



Carries out registration of the records DNS in the corresponding zones relating to a jail. For its correct work correctly adjusted DNS server and a key is necessary for zone updating.
As zones interested in updating at start and a stop of a jails can be a little, it is possible to list them in rc.conf of jails.

When parameters ddns_zone_list not empty, at performance jstart и jstop, command


              
cbsd ddns mode=add jname=xxx


              
and


              
cbsd ddns mode=delete jname=xxx


              
are carried out automatically.
Argument mode means carried-out action with record - can be add or delete. 

The minimum configuration in jail rc.conf:


              
ddns_zone_list="";


              

should have the list of names of configurations for zones. One record for one zone. In a name there should not be points.



              
ddns_key_name_of_zone="";


              

The parameter contains a way to the file of a private key of a zone name_of_zone, listed in ddns_zone_list.



              
ddns_zones_name_of_zone="";


              

contains the list of those records which should be updated in the corresponding zone.



Example: configuration for updating of two zones: my.domain and bsdstore.ru and addition or removal
Jail IP addresses under the records test1.my.domain + test2.my.domain and relay.bsdstore.ru:




              
ddns_zone_list="my_domain bsdstore";
ddns_key_my_domain="/usr/jails/etc/zonekeys/Kmy.domain.+157+52142.private";
ddns_key_bsdstore="/usr/jails/etc/zonekeys/Kbsdstore.+157+52142.private";
ddns_zones_my_domain="jail1.my.domain jail1.my.domain";
ddns_zones_bsdstore="relay.bsdstore.ru";


              

Configuration for DNS with named:


              
key bsdstore.ru. {algorithm "HMAC-MD5";secret "YrVW9yP6gNMA7VbcU/r2mSIwYnFj/XkCDd6QuqOHE26/ipnrPy+eXrKrUyaFhB2XWNdVLUX7QCUkfhg4zN5YiA==";};
zone "bsdstore.ru" {type master;file "/etc/namedb/dynamic/bsdstore.ru";allow-update {key bsdstore.ru; };};


              

key and private key generated by:


              
dnssec-keygen -b 512 -a HMAC-MD5 -v 2 -n HOST bsdstore.ru.


              




              
Searching for jail in node farm

              
Commands:



              
% cbsd jwhereis
% cbsd jailmapdb


              

Description:



In the case of a few FreeBSD/cbsd node farm, you can generate a map of location jails on the appropriate servers.
This functionality uses a utility jlogin search for remote jails. Also, this map can use a variety of tools, such as
to custom admin panel, for auto-documentation of the farm, as well as a variety of services (for example, Bacula jail can automatically search for it on the map
new jail and to create for them a configuration for backup; or when jail migrate from one physical node to another, reconfigure
the target host with jails for backup without system administrator manual action).