Developer Manual¶

Introduction¶
NethServer is an Open Source operating system for the Linux enthusiast, designed for small offices and medium enterprises. It’s simple, secure and flexible.
NethServer is ready to deliver your messages, to protect your network with the built-in firewall, share your files and much more, everything on the same system.
Target audience¶
This manual is intended for developers who are interested about NethServer internals. Reading this manual you should be able to learn how to extend, enpower and debug the NethServer platform.
Get involved¶
The NethServer project welcomes anyone who would like to become involved in the project. This is a brief list of things to do (in a sparse order):
- Writing documentation
- Bug reporting, bug triaging, QA testing
- Translating the web interface
- Developing / coding in Perl, PHP, Python, Bash
Note
This manual is a work in progress. If you can’t find some information please check also our wiki: http://wiki.nethserver.org and always feel free to ask on http://community.nethserver.org
Development process¶
Issues¶
An issue is a formal description of a known problem, or desired feature, inside a tracker. There are two kind of issues:
- Bug
- describes a defect of the software; it must lead to a resolution of the problem. For example, a process crashing under certain conditions.
- Feature/Enhancement
- describes an improvement of the current code or an entirely new feature. For example, remove a harmless warning of a running process or designing a new UI panel.
Bugs and enhancements will always produce some code changes inside one or more git repositories.
Each repository is associated with one or more RPM packages. Changes to the code produce new releases of RPM packages.
Do I need to open a new issue?¶
Yes, if what you’re talking about will produce some code. By the way, it’s perfectly reasonable not to fill issues for occasional small fixes, like typos in translations.
When implementing small fixes, always avoid commits to the master branch. Open a Pull requests and carefully describe the problem. Creation of issues can be avoided only for trivial fixes which require no QA effort.
Issues are not a TODO list. Issues track the status changes of a job, the output of the job will be a new RPM resolving the issue itself. If you are exploring some esoteric paths for new feature or hunting something like a heisenbug , please write a draft wiki page with your thoughts, then create a new issue only when you’re ready to write a formal description and produce some output object.
A process for a new feature should be something like this:
- Open a new topic on http://community.nethserver.org and discuss it.
- If the feature is complex, a dedicated wiki page could be written on
https://github.com/NethServer/dev/wiki (or http://wiki.nethserver.org/).
- Create a wiki page with notes and thoughts (team contributions are welcome!).
- If the wiki page is a feature design document, the feature can simply point to the wiki page.
- The wiki page should become a changelog for a new release.
- When the wiki page is pretty “stable” and the whole thing is well outlined, a team member will create one or more new issues.
- Open the issue on GitHub https://github.com/NethServer/dev/issues/new.
Writing issues¶
How to write a bug report:
- Apply the bug label
- Point to the right software component with the associated version
- Describe the error, and how to reproduce it
- Describe the expected behavior
- If possible, suggest a fix or workaround
- If possible, add a piece of system output (log, command, etc)
- Text presentation matters: it makes the whole report more readable and understandable
How to write a feature or enhancement:
- Everybody should understand what you’re talking about: describe the feature with simple words using examples
- If possible, add links to external documentation
- Text presentation and images matter: they make the whole report more readable and understandable
More information:
Issue priority¶
Bugs should always have priority over features.
The priority of a bug depends on:
- security: if it’s a security bug, it should have maximum priority
- number of affected users: more affected users means more priority
Issue tracker¶
The NethServer project is hosted on GitHub and is constituted by many git repositories. We set one of them to be the issue tracker:
https://github.com/NethServer/dev
Issues created on dev help coordinating the development process, determining who is in charge of what.
Issue labels¶
Issues can be tagged using a set of well-known labels:
- bug: a defect of the software
- testing: packages are available from testing repositories (see QA team member (testing))
- verified: all test cases were verified successfully (see QA team member (testing))
- invalid: invalid issue, not a bug, duplicate or wontfix. Add a detailed description and link to other related issue when using this tag.
An issue without a label is considered a new feature or an enhancement.
Until June 2024, both NethServer 7 (NS7) and NethServer 8 (NS8) are actively supported. You can use the ns7 label label to mark the issues for NethServer 7.
Before introducing new labels, please discuss them with the development team and make sure to describe carefully the new label inside the label page.
Developer¶
The Developer.
- Sets the Assignee to himself.
- Bundle his commits as one or more GitHub Pull requests
- For enhancements, writes the test case (for bugs the procedure to reproduce the problem should be already set).
- Writes and updates the Documentation associated with the code.
- Finally, clears the Assignee.
If the issue is not valid, it must be closed using the invalid label. A comment must convey the reason why it is invalid, like “duplicate of (URL of issue), wontfix because …”.
QA team member (testing)¶
The QA team member.
- Takes an unassigned issue with label testing and sets the Assignee field to him/herself.
- Tests the package, following the test case documentation written by the Developer.
- When test finishes he/she removes the testing label and clears the Assignee field. If the test is successful, he/she sets the verified label, otherwise he/she alerts the Developer and the Packager to plan a new process iteration.
Packager¶
The Packager coordinates the Developer and QA member work. After the Developer opens one or more pull requests:
- Selects issues with open pull requests
- Reviews the pull request code and merges it
- Builds and uploads the RPMs to the testing repository and sets the testing label (see Building RPMs)
After the QA member has completed the testing phase:
- Takes an unassigned issue with label verified
- Commits a release tag (see Building RPMs).
- Re-builds the tagged RPM.
- Uploads the RPM to updates (or base) repository.
- Pushes the release tag and commits to GitHub
- Closes the issue, specifying the list of released RPMs
When the package is CLOSED, all related documentation must be in place.
Pull requests¶
A Pull Request (PR) is the main method of submitting code contributions to NethServer.
You can find an overview of the whole workflow here: https://guides.github.com/introduction/flow/
Submitting a pull request¶
When submitting a PR, check that:
- PR is submitted against
master
(for current stable release) - PR title contains a brief explanation of the feature, fix or enhancement
- PR comment contains a link to the related issue, in the form
NethServer/dev#<number>
, eg: NethServer/dev#1122 - PR comment describes the changes and how the feature is supposed to work
- Multiple dependent PRs in multiple repositories must include the dependencies between them in the description
- Select at least one PR reviewer (GitHub suggestions are a usually good)
- Select yourself as the initial PR assignee
Managing an open pull request¶
After submitting a PR, before it is merged:
- If enabled, automated build process must pass
- If the build fails, check the error and try to narrow down the reason
- If the failure is due to an infrastructure problem, please contact a developer who will help you
- Another developer must review the pull request to make sure it:
- Works as expected
- Doesn’t break existing stuff
- The code is reasonably readable by others developers
- The commit history is clean and adheres to Commit message style guide
- The PR must be approved by a developer with commit access to NethServer on GitHub:
- Any comment raised by a developer has been addressed before the pull request is ready to merge
Merging a pull request¶
When merging a PR, make sure to copy the issue reference inside the merge commit comment body, this step will be used by automation tools:
- to write notification about published RPMs inside the referenced issue
- to automatically create RPMs changelog
If the commit history is not clear enough, or you want to easily revert the whole work, it’s acceptable to squash before merge. Please make sure the issue reference is present inside the comment of the squashed commit.
Also, avoid adding the issue references directly inside non-merge commit messages to have a clean GitHub reference graph.
Example of a good merge commit:
commit xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Merge: xxxxxxx yyyyyyy
Author: Mighty Developer <mighty.developer@netheserver.org>
Date: Thu Dec 14 17:12:19 2017 +0100
Merge pull request #87 from OtherDev/branchXY
Add new excellent feature
NethServer/dev#1122
Example of a merged PR with squash:
commit xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Author: Mighty Developer <mighty.developer@netheserver.org>
Date: Thu Dec 14 17:12:19 2017 +0100
Another feature (#89)
NethServer/dev#1133
Draft pull requests¶
The use of draft pull requests is recommended to share an on-going development. Draft pull requests can be used to test possible implementations of features that do not have an issue yet. If the draft pull request does not reference an issue it must have an assignee.
RPM Version numbering rules¶
NethServer releases carry the version number of the underlying CentOS.
For example NethServer 7 beta1
is based on CentOS 7
.
Packages have a version number in the form X.Y.Z-N (Eg.
nethserver-myservice-1.0.3-1.ns7.rpm
):
- X: major release, breaks retro-compatibility
- Y: minor release, new features - big enhancements
- Z: bug fixes - small enhancements
- N: spec modifications inside the current release - hotfixes
Commit message style guide¶
Individual commits should contain a cohesive set of changes to the code. These seven rules summarize how a good commit message should be composed.
- Separate subject from body with a blank line
- Limit the subject line to 50 characters
- Capitalize the subject line
- Do not end the subject line with a period
- Use the imperative mood in the subject line
- Wrap the body at 72 characters
- Use the body to explain what and why vs. how
For merge commits, and commits pushed directly to master branch (avoid whenever possible!), also add the issue reference inside the commit body.
Documentation¶
The developer must take care to write all documentation on:
- wiki page during development
- Developer Manual and/or README.rst before release
- Administrator Manual before release
- Inline help before release
Packages should be inside the testing or nethforge-testing repositories until all documentation is completed.
New packages¶
Before creating a new package, make sure it’s a good idea. Often a simple documentation page is enough, and it requires much less effort. When trying new things, just take care to write down on a public temporary document (maybe a wiki page) all steps and comments. If the feature collects many requests, it’s time to think about a new package. Otherwise, the temporary document can be moved to a manual page.
When creating a new package, make sure the following requirements are met:
- Announce it on http://community.nethserver.org
- Create an issue describing the package
- Create a personal repository on GitHub
- Add a GPL license and copyright notice in the COPYING file
- Add a README.rst file, with developer documentation
- If needed, create a pull request for the NethServer/comps or NethServer/nethforge-comps repository, to list the package in the Software center page.
- Build the package and push it to testing or nethforge-testing repository
See also Building RPMs.
Package updates¶
Updates to RPM packages must obey the following rules:
- New features/enhancements and bug fixes must not alter the behavior of existing systems
- New behaviors must be enabled by an explicit and documented sysadmin operation
- RPM packages must support updates from any previous release of the same branch
Minor release from upstream¶
On every upstream (CentOS) minor release, the QA team should check the following hot points before clearing the way to the new release:
- Samba: authenticated access to file shares
- Mail server
- sending and receiving mail
- antivirus filter on received mail
- Groupware
- basic mail features
- access to calendars and contacts
- Web proxy
- web access with transparent proxy
- antivirus filter on using EICAR test
- Web applications (eg. NextCloud)
- Asterisk with dahdi kernel modules and FreePBX
ISO releases¶
Usually, the NethServer project releases a new ISO image in the following cases:
- when the upstream project releases a new ISO image. The NethServer ISO is rebased on it.
- when packages bundled in the ISO receive new features that affect the installation procedure and/or the initial system configuration.
The NethServer ISO is almost equivalent to the upstream one, except for the following points:
- Additional boot menu options and graphics
- Additional Anaconda kickstart scripts and graphics
- Additional RPMs from the NethServer project
See also Building ISO.
Pre-releases¶
Before any final ISO release, the software development process goes through some test versions, usually called alpha, beta and release candidate (RC). These releases are an excellent way to experiment with new features, but may require some experience using a Linux system and/or the command line.
Alpha releases are not ready to be used in production because some features are not finished, furthermore upgrade to the final release will not be supported (but may be possible).
Beta releases could be used in production, especially if new features are not used on mission-critical systems. Upgrades to the final release are supported.
Release candidates (RC) can be run in production, all features are supposed to be complete and bug-free. The upgrade to the final release will be minor or less.
RPM package rules¶
Naming and events conventions¶
Each package name MUST be composed of
- a prefix, corresponding to the product name: nethserver-, neth-, ….
- the feature/function/daemon/software: base, directory, httpd-admin …
Each package MUST contain a <packagename>-update
event, raised each time
the package is installed/updated and when the system is re-configured (for instance,
after another package has been uninstalled). The update event should
- configure the package on first install
- take care of upgrading current installation in case of package update
Note
You should not add code in %post
and %pre
sections of the spec file.
All the logic must be inside the -update
event.
Each package MAY contain a <packagename>-save
event, raised by the console
or the web interface to adjust the package configuration after some DB value has changed.
For example, given a package named nethserver-dnsmasq:
- update event:
nethserver-dnmasq-update
- save event:
nethserver-dnmasq-save
Install/Update process¶
Just after a package transaction (install/update), the NethServer yum plugin will:
- execute all nethserver-update-<package>: events of installed/updated packages in the current transaction
- execute
runlevel-adjust
event to start/stop all configured services - execute
firewall-adjust
event to open/close firewall ports
In case of manual installation , the update, firewall-adjust and
runlevel-adjust events must be fired manually using the signal-event
command.
Uninstall process¶
After a package is removed, the NethServer yum plugin will:
- execute all nethserver-update-package event of installed packages
- execute runlevel-adjust event to start/stop all configured services
- execute firewall-adjust event to open/close firewall ports
Service packages¶
A service package is an RPM which is responsible for a system service configuration and management. The package must follow all rules listed above plus some more conventions.
Configuration DB default¶
Mandatory:
- type: service
- status: the current service status, can be enabled or disabled
Optional:
- TCPPort: a tcp listening port
- UDPPort: a udp listening port
- TCPPorts: a list of tcp listen ports. Eg: 123,678
- UDPPorts: a list of udp listen ports. Eg: 123,678
For example, the package nethserver-puppet managing the service puppet will contain:
- /etc/e-smith/db/configuration/defaults/puppet/type
- /etc/e-smith/db/configuration/defaults/puppet/status
Beside this, each packge MUST always declare its own set of keys and properties inside default databases.
Events and actions¶
nethserver-base
package provides two generic events:
runlevel-adjust
: enable/disable the service using chkconfig command and start/stop the servicefirewall-adjust
: read TCPPort(s) and UDPPort(s) props and open the specified ports in the firewall configuration
Both events are handled by the system, so there is no need to link these events into the package.
Further documentation: perldoc /usr/share/perl5/vendor_perl/NethServer/Service.pm
Orphan services¶
During runlevel-adjust event, the system will stop any orphan service. A orphan service is a running service not controlled by any nethserver-package. A service is an orphan if there is a service record in configuration db, and there is no db defaults (in /etc/e-smith/db/configuration/defaults).
Internationalization¶
These are the coding conventions for NethServer i18n. Each package repository should respect them.
The developer prepares the translation source strings when the code has been written.
Each translation catalog must be mapped to a resource on Transifex.
Whenever new strings are added or existing ones are changed, source catalogs must be pushed into Transifex with Transifex client:
tx push -s
Once translators complete their work a new release of nethserver-lang can occur.
Currently both gettext and Server Manager specific formats are supported for language catalogs.
Server Manager (Cockpit)¶
Packages providing an application for the Cockpit Server Manager must adhere to the guidelines documented here:
https://nethserver.github.io/nethserver-cockpit/i18n/#translations
Server Manager (Nethgui)¶
To configure the Transifex client and find the string catalog files execute the txinit.sh script on the repository root. The script can run multiple times, when new catalogs are added to the repository.
PHP files¶
All source language catalog files are placed in
root/usr/share/nethesis/NethServer/Language/en
. We assume
en
is en-US
.
The catalog format is a common PHP file where the array variable
$L
is assigned some keys:
<?php
$L['string_id_1'] = 'This is an example';
$L['string_id_2'] = 'What is a PHP array catalog?';
- Omit ending sequence
?>
. - Use UTF-8 encoding.
Given a module with name “Test”, the source language file will be
root/usr/share/nethesis/NethServer/Language/en/NethServer_Module_Test.php
.
When the Server Manager is running in debug mode, missing translation
labels can be found in /var/log/messages
. To enable the
debug,
Unlock
index_dev.php
controller:touch /usr/share/nethesis/nethserver-manager/debug
Prepend
index_dev.php
on URLs path, eg:https://<ipaddress>/index_dev.php/en/<module>
.
gettext files¶
The gettext format is used by the TODO scripts in the old Server Manager.
The xgettext
command can be used to extract strings from the
source code. The resulting .pot
file must be named
locale/<rpmname>.pot
.
For example we will translate a new TODO message for the web interface.
Steps to setup translation for a new todo:
Be sure gettext is installed in your system
Add the todo script to the git repository
Create locale directory inside the git repository root:
mkdir locale/
Extract the strings (from a Python source code):
PACKAGE=$(basename `pwd`) xgettext --msgid-bugs-address "translations@nethserver.org" --package-version "1.0.0" \ --package-name "$PACKAGE" --foreign-user -d "$PACKAGE" -o "locale/${PACKAGE}.pot" \ -L Python root/etc/nethserver/todos.d/*
Run the txinit.sh script and commit newly created files into git.
Upload the new
.pot
resource to Transifex (special permissions are needed):tx push -s
Databases¶
Overview¶
All user-editable configuration parameters on NethServer are stored in plain text database.
These values are used to generate the system configuration files, such
as those found in the /etc/
directory.
The configuration databases may be modified by various programs on the
system, including the web interface or scripts run from the command line
by a system administrator.
Each entry in the database is either a simple key/value pair or a key and a collection of related property/value pairs.
Simple entries¶
Simple configuration database entries take the form of a key/value pair:
[root@nsrv -]# config show SystemName
SystemName=myserver
Complex entries¶
More complex entries consist of a key, a type, and a collection of property/value pairs:
[root@nsrv -]# config show sysconfig
sysconfig=configuration
Copyright=
ProductName=NethServer
Registration=none
Release=4
Version=6.4
Use complex entries whenever possible
Access from the command line¶
You can access database entries from the command line using the
config
command, as shown above, or the db
command. The
config
command provides a shorthand for accessing the
configuration database. The following commands are equivalent:
[root@nsrv -]# config show SystemName
SystemName=nsrv
[root@nsrv -]# db configuration show SystemName
SystemName=nsrv
The db
command allows you to access all of the databases. For
example to show the details of the test entry from accounts db:
[root@nsrv -]# db accounts show test
test=user
City=
Company=
Department=
FirstName=test
LastName=test
PhoneNumber=
Street=
Uid=5000
VPNClientAccess=yes
VPNRemoteNetmask=255.255.255.0
VPNRemoteNetwork=192.168.1.0
For more options see help of db
command:
db -h
Access via the Perl API¶
You can also access configuration database entries programmatically
using the esmith::ConfigDB
and related Perl modules, which are
abstractions for the esmith::DB
module.
For example, we can retrieve and show the admin account details like
this:
use esmith::AccountsDB;
my $db = esmith::AccountsDB->open or die "Couldn't open AccountsDB\n";
my $admin = $db->get("admin") or die "admin account missing from AccountsDB\n";
print $admin->show();
For documentation on Perl API use the perldoc
command. Eg:
perldoc esmith::ConfigDB
Database initialization¶
The configuration databases are initialized from files in the
/etc/e-smith/db/
hierarchy. These files can perform one of three
actions:
- Create a database entry and set it to a default value, if the entry does not already exist.
- Migrate an entry from a previous value to a new value.
- Force a database entry to a specific value, regardless of its current setting (use with care!)
This design allows each package to provide part of the system configuration, or migrate the system configuration values as required. Note that a single database property can only be owned by one package. Database initialization is run during system install, system upgrade and after new software has been installed.
If you examine the /etc/e-smith/db/configuration/
directory you will
see three subdirectories: defaults/
, force/
and migrate/
to
match the three options above. A similar structure exists for each of
the other databases. A new database can be created by populating a new
directory tree under the /etc/e-smith/db/
directory.
Configuration databases can also be initialized using a special
/usr/libexec/nethserver/initialize-<dbname>-database
script, where dbname is the database name.
For example: /usr/libexec/nethserver/initialize-mycustomdb-database
.
Defaults files¶
Defaults files are simple text files. If the corresponding database key/property already exists, it is skipped. Otherwise, the key/property is created and the value loaded. For example, this file:
[root@nsrv -]# cat /etc/e-smith/db/configuration/defaults/sshd/status
enabled
It would create the sshd
database entry if it doesn’t already exist,
create the status
property for that entry, again if it doesn’t
already exist, and finally set the status property to enabled
.
Forcing database initialization¶
Simply call the action:
/etc/e-smith/events/actions/initialize-default-databases
Force files¶
Force files are just like defaults files, except they overwrite the existing value. So, this file:
[root@nsrv -]# cat /etc/e-smith/db/configuration/force/sysconfig/Version
6
It would create the Version property of the sysconfig entry and
unconditionally set its value to 6
.
Warning
Do not use force fragments if not really necessary!
Migrate fragments¶
Migrate fragments are small pieces of Perl text which can be used to perform more complex migrations than is possible with defaults and force files. They would normally be used to replace database keys or properties with new names, or to adjust policy settings during an upgrade.
Each fragment is passed a reference to the current database in the
$DB
variable. This variable is an instance of the appropriate
esmith::DB subclass, e.g. esmith::AccountsDB
when the accounts
database migrate fragments are being executed. This means that you can
use the methods of that subclass, for example
esmith::AccountsDB->users()
.
Here is an example of a migrate fragment, which replaces the outdated
popd
entry with the new name pop3
:
{
my $popd = $DB->get("popd") or return;
my $pop3 = $DB->get("pop3") || $DB->new_record("pop3", { type => "service" });
$pop3->merge_props($popd->props);
$popd->delete;
}
This fragment checks whether the database (the configuration database in
this case) has a popd
entry. If that entry does not exist, the
migrate fragment returns immediately. If the popd
entry exists, we
need to convert it, so we retrieve the pop3
entry (or create it if
it doesn’t already exist). We then merge the properties from the
popd
entry into the pop3
entry and finally delete the popd
entry.
If this migrate fragment is run again, it will return immediately as the
popd
entry has already been deleted.
Important notes about migrate fragments¶
- Please be careful with migrate fragments. Although they should only modify entries within the current database, there are no restrictions placed on what they can do. The ability to open and even modify other databases may be required to perform a migration.
- Migrate fragments must be safe to run multiple times. They should migrate the value when required and do nothing in other cases.
- Migrate fragments should never call
croak
ordie
. This will cause the database migration to stop. If an error is detected, callcarp
orwarn
to note the error in the logs. - Migrate fragments should call good termination with
return(0)
rather thanexit(0)
. - Migrate fragments should be owned by the package requiring the migration so that the migration only occurs when that package is installed.
- Migrate fragments should be self-contained and ideally perform only one migration per fragment.
- DO NOT USE to initialize default database values.
Evaluation order¶
When a database is loaded:
- migrate scripts are run first
- then defaults are loaded
- and finally any force files are loaded.
This order allows migration of old format entries to occur prior to loading of new default values. Remember, defaults will not change an existing database property.
Best practices¶
- The configuration databases should only be modified using the tools and APIs provided.
- The order of the entries and the order of properties is undefined.
- The keys and property names are currently treated in a case-sensitive manner, though this may change in the future. Do not create keys or property names which differ only by their case.
- Underscores and hyphens are valid in key and property names, but should normally be avoided.
- Do not “overload” an existing property with a new value. If the existing values do not meet your requirements, discuss your implementation with the developers. Values which are not known by the base may cause serious issues on upgrade. If the existing panels have three choices, do not invent new choices without enhancing the panel to support them.
- The
type
pseudo-property is used internally and is reserved . - By convention, database keys are lower case, and property names are
stored in mixed case. The
type
,status
andaccess
properties are exceptions to this convention. - The storage location and internals of the databases is subject to change.
- The configuration databases are currently stored as pipe-delimited
flat text files in the
/var/lib/nethserver/db/
directory.
Namespace issues¶
All entries in a single database share the same namespace. Users, groups, information bays, printers, and other entries in the accounts database currently all share one namespace. This means that you cannot have a user with the same name as an information bay, group or other entry in the accounts database.
However, it would be possible to have a host named fredfrog as well as a user named fredfrog as they are stored in separate databases and thus different namespaces.
List of available database¶
Table of databases¶
The following table summarizes
- the database name
- the perl module that manages it and
- the package that provides it
Databases provides by the base system:
Name | Perl module | Package | Description |
---|---|---|---|
configuration | esmith::ConfigDB | nethserver-base | |
hosts | esmith::HostsDB | nethserver-hosts | |
networks | esmith::NetworksDB | nethserver-base | |
domains | esmith::DomainsDB | nethserver-mail-common |
Each modules can define its own new databases. Some relevant databases are:
Name | Perl module | Package | Description |
---|---|---|---|
accounts | esmith::AccountsDB | nethserver-directory | |
domains | esmith::DomainsDB | nethserver-mail-common |
Templates¶
Design of the template system¶
Every piece of software has its own configuration format, and writing parsers for each one is a complex, time-consuming and error-prone process. The template system software avoids the whole issue by using templates which generate the correct configuration.
Configuration files are over-written when templates are expanded. In a few specific cases, the existing configuration file is parsed and rewritten in-place. This is done where the configuration file is also automatically updated by some other process.
Templates are stored under /etc/e-smith/templates/
in a directory
hierarchy which matches the standard filesystem. For example, the
template for /etc/inittab
is stored in the
/etc/e-smith/templates/etc/inittab/
directory. Each template is stored
as a directory of template fragments and processed by the Perl
Text::Template
module.
The template fragments are concatenated together in ASCIIbetical order (US-ASCII sort order) and the complete file is parsed to generate the appropriate configuration files for the service. The use of fragments is part of NethServer modular and extensible architecture; it allows third-party modules to add fragments to the configuration where necessary.
The Text::Template module¶
The Text::Template
module allows arbitary Perl code to be embedded in
a template file by surrounding it in curly braces ({
and }
). The code
inside the braces is interpreted and its return value replaces the
section between, and including, the braces. For instance:
The answer is { 40 + 2 }
becomes:
The answer is 42
Variables can be passed in from the program which is expanding the template, hence:
{
$OUT = ';'
for my $item ( qw(bread milk bananas) )
{
$OUT .= "\* $item\n";
}
}
would expand to:
Shopping list
* bread
* milk
* bananas
The template system uses this mechanism to automatically pass in global configuration variables from the configuration database which can then be used to fill out the configuration files.
For example, the /etc/hosts
template could be fairly simple and composed of
two fragments:
[root@test hosts]$ ls /etc/e-smith/templates/etc/hosts
10localhost 20hostname
Fragments can have static content. For example, the first fragment:
127.0.0.1 localhost
The second is more complex and relies on values from the configuration database:
{
$OUT .= "$LocalIP\t";
$OUT .= " ${SystemName}.${DomainName}";
$OUT .= " ${SystemName}";
}
Note that the whole fragment is enclosed in braces. Within those braces is a section of Perl code.
When this template is expanded, it results in the following configuration file:
# ================= DO NOT MODIFY THIS FILE =================
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at https://dev.nethesis.it/projects/nethserver/wiki/NethServer
# original work from http://www.contribs.org/development/
#
# Copyright (C) 2015 Nethesis S.r.l.
# http://www.nethesis.it - support@nethesis.it
#
127.0.0.1 localhost
192.168.10.1 nsrv.nethesis.it nsrv
The header block comes “for free” as part of the template system,
courtesy of an optional file template-begin
, which is always processed
as the first fragment. If it isn’t provided, the text shown with #
comments is included.
If target configuration file do not support line comment beginning with #,
please provide a custom or empty template-begin
.
The other lines are provided by the two fragments shown above. Note the
use of the configuration database variables: $LocalIP
, $SystemName
and $DomainName
. All simple entries in the configuration database are
provided as global variables to the templates.
Note that all of the template fragments are concatenated together before evaluation, so it is possible to set values in fragments which are used in later fragments. This is a very useful model for reducing the code in individual template fragments.
The complex entries in the configuration database are also provided as
global variables to the templates. However, they are provided as Perl
hashes instead of simple scalars. For example, here is how you might
configure the Network Time Protocol (NTP) server /etc/ntp.conf
file:
server { $ntpd{NTPServer} }
driftfile /etc/ntp/drift
authenticate no
The NTPServer setting is stored in the ntpd configuration database
record, and so can be accessed via the hash accessor $ntpd{NTPServer}
.
template-begin and template-end¶
Each template directory can contain two optional files template-begin
and template-end
. The template-begin file is always processed as the
first file of the template, and the template-end file is always
processed as the last file.
If the directory does not contain a template-begin
file, the contents
of /etc/e-smith/templates-default/template-begin
is used
automatically.
If the directory does not contain a template-end
, nothing is appended
to the template output. It is mostly used to provide the closing block
for configuration files written in languages such as HTML and PHP,
through a link to an entry in the templates-default/
directory.
/etc/e-smith/templates-default¶
The /etc/e-smith/templates-default
directory contains a set of
template-begin and template-end files for various languages. For
example, if your template generates a perl script, you would link
template-begin
to /etc/e-smith/templates-default/template-begin-perl
and automatically get the #!/usr/bin/perl -w
line and a comment
containing the contents of the default template-begin file.
Note
You may also need a templates.metadata
configuration file if your generated file needs to be executable.
Template fragment ordering¶
Template fragments are assembled in ASCII-betical order, with two exceptions: template-begin always comes first, and template-end always comes last. Template fragments are often named to start with a two digit number to make the ordering obvious, but this is not required.
Templates for user home directories: templates-user¶
Most of the templates on the system map to single, fixed output files,
such as /etc/hosts
. However, templates are also used to generate
configuration files such as mail delivery instructions for users. These
templates are stored in the /etc/e-smith/template-user/
tree.
As these templates have a variable output filename, they are expanded using small pieces of Perl code in action scripts.
Local site overrides: templates-custom and templates-user-custom¶
It is possible that the standard templates are not correct for a
particular installation, and so the local system administrator can
override the existing templates by placing files in the
templates-custom
tree. This is a parallel tree to the normal templates
hierarchy, and is normally empty. There is also a template-user-custom
tree for overriding entries in the templates-user tree.
Be aware of overwriting all settings of a template if you copy the whole template to custom templates.
This means, if there is an update at the original template it is overritten with your old version at custom template.
If you only want to add a line to the config create an empty custom template and do it there.
If a templates-custom entry exists for a template, it is merged with the standard templates directory during template expansion, using the following rules:
- If a fragment of the same name exists in both templates and templates-custom, the one from templates-custom is used, and the one from the standard templates tree is ignored.
- If the fragments in templates-custom have different names from those in templates, they are merged into the template as if they were in the templates directory.
- If the templates-custom entry is a file, rather than a directory, it completely overrides the standard template.
To make this concrete, let’s assume we have the following template
structure in /etc/e-smith/templates/etc/book.conf
:
10intro
30chapter3
40chapter4
80synopsis
and in /etc/e-smith/templates-custom/etc/book.conf
:
30chapter3
50chapter5
The resulting template would be processed in this order:
- template-begin from /etc/e-smith/templates-default
- 10intro from /etc/e-smith/templates/etc/book.conf
- 30chapter3 from /etc/e-smith/templates-custom/etc/book.conf
- 40chapter4 from /etc/e-smith/templates/etc/book.conf
- 50chapter5 from /etc/e-smith/templates-custom/etc/book.conf
- 80synopsis from /etc/e-smith/templates/etc/book.conf
- template-end (empty), nominally from /etc/e-smith/templates-default
How to resolve conflicts with standard templates¶
It is possible that the standard templates may specify behaviour which is not appropriate for your application. In many cases the templates will be driven by configuration database settings which allow their behaviour to be customized, which should be the first thing to check.
In many cases, your application only needs to extend the behaviour of the template by adding one or more fragments. This should be your second option and can be achieved by simply adding your fragment in the correct place in the list of fragments.
In rare cases the standard template specifies a behaviour which conflicts with your application. In these cases, you should do all of the following:
- Create a templates-custom directory to match the existing one in the templates hierachy.
- Copy the conflicting fragment, and only that fragment, to the templates-custom directory. The fragment should have the same name in both directories. At this point you have not changed the behaviour of the system as the templates-custom entry will be preferred, but will behave identically.
- Modify the copy in templates-custom to suit your required behaviour.
- Inform the NethServer team about the problem. Please attach your modified template (or even better, a patch file) and provide details of why you think that the standard template should be changed.
The expansion of templates¶
To expand your custom templates to their destination you have to use the following command:
expand-template <template destination>
where template destination has to be changed with the true path to the configuration file.
For Example you want to add something to the samba configuration,
then you have to build a custom template fragment under
/etc/e-smith/template-custom/etc/samba/smb.conf/
directory
and execute the command:
expand-template /etc/samba/smb.conf
Subdirectory templates¶
It is also possible to split templates into further subdirectories. This
can be very useful for evaluating the same fragments in a loop, for
example for each virtual domain in httpd.conf
or each ibay in
smb.conf
.
Two examples of this can be found in
/etc/e-smith/templates/etc/httpd/conf/httpd.conf/80VirtualHosts
which
loops over the
/etc/e-smith/templates/etc/httpd/conf/httpd.conf/VirtualHosts/
directory, and /etc/e-smith/templates/etc/smb.conf/90ibays
which
performs a similar loop over the
/etc/e-smith/templates/etc/smb.conf/ibays/
directory.
Template expansion¶
The system is designed to ensure consistent and reliable operation, without requiring command-line access. Whenever an event is signalled, the relevant templates for that event are expanded and the services are notified of the configuration changes.
Requesting expansion of a template in an event is a simple matter of
creating an empty file under the templates2expand
hierarchy for that
event.
See events manual chapter for further information.
Template permissions and ownership: templates.metadata¶
Templates are normally expanded to be owned by root
and are not
executable, which is a reasonable default for most configuration files.
However, templates may need to generate configuration files which are
owned by a different user, or which need to be executable or have other
special permissions. This can be done by creating a templates.metadata
file which defines the additional attributes for the expansion.
Note
Configuration files should generally not be writable
by any user other than root. In particular, configuration files should
not normally be writable by the www user as this poses a significant
security risk. Installation advice which says chmod 777
is almost
invariably wrong.
For example, here is the metadata file
/etc/e-smith/templates.metadata/etc/ppp/ip-up.local
:
UID="root"
GID="daemon"
PERMS=0755
which sets the group to daemon
and makes the script executable. Note
that the file is readable by members of the daemon
group, but it is
not writable by anyone but root. It is also possible to use the same
template to generate multiple output files, such as in this example:
TEMPLATE_PATH="/etc/sysconfig/network-scripts/route-ethX"
OUTPUT_FILENAME="/etc/sysconfig/network-scripts/route-eth1"
MORE_DATA={ THIS_DEVICE => "eth1" }
FILTER=sub { $_[0] =~ /^#/ ? '' : $_[0] } # Remove comments
The templates.metadata file for route-eth0 just uses eth0
instead of
eth1
on the second and third lines. Note also the FILTER
setting
which allows post-processing of the generated template.
There are many examples under /etc/e-smith/templates.metadata/
and the
full list of options can be seen with:
perldoc esmith::templates
Template deletion: templates.metadata¶
A template once expanded in the file system cannot be deleted automatically,
you can only change its content. The template will still exist until you delete
it by a templates.metadata
file.
For example to delete /etc/myExample
, create the file
/etc/e-smith/templates.metadata/etc/myExample
with just the following line
DELETE=1
Then
expand-template /etc/myExample
Perl API: processTemplate¶
In rare circumstances you may need to call processTemplate
directly.
Explicit calls to processTemplate
are typically only used when the
output filename is variable:
use esmith::templates;
foreach my $name (@names)
{
[...]
processTemplate(
{
TEMPLATE_PATH => "/etc/myservice/user.conf",
OUTPUT_FILENAME => "/etc/myservice/$name.conf"
);
[...]
}
bq. Content is available under GNU Free Documentation License 1.3 unless otherwise noted. Original document from: http://wiki.contribs.org/
Actions and events¶
Actions¶
An action is a program, frequently written in a scripting language, which performs a single task. It is typically an encapsulation of a task usually done by a system administrator, such as editing a configuration file or reconfiguring a service. Actions are not called directly; they are always called by signalling an event.
The actions are stored in the /etc/e-smith/events/actions/
directory.
These actions are then linked into the relevant events as the same
action may need to be performed in more than one event.
To create a new action called myaction you simply create a program
to perform the action myaction and save it as
/etc/e-smith/events/actions/myaction
. Actions can be written in any
programming language, although additional platform support is provided
for Perl code.
An example action script is:
#!/bin/bash
/usr/sbin/lokkit --update
Action script parameters¶
Action scripts are always called with at least one parameter; the name of the current event. Many action scripts can be called with a single additional parameter. This parameter is usually a configuration database key, for example the username being modified. Action scripts rarely require more than two parameters. The details should be stored in the configuration database(s) and only the key should be passed to the action scripts. All configuration details must be stored in the configuration databases and the database key passed as the parameter to the action. This allows other scripts to be added to the event.
Since the system passes the name of the current event as the first parameter, it is often beneficial to write action scripts which are polymorphic based on the event name. For example, the code to create a user and the code to modify an existing user may be only slightly different and may well benefit from being in a single script. Example:
use strict;
my $event = $ARGV[0];
my $myarg = $ARGV[1];
exit 0;
Note
Whenever possible, avoid to call events from within action scripts.
Action code libraries¶
To promote code reusability and components abstraction some Perl
modules are available under
/usr/share/perl5/vendor_perl/NethServer/
and
/usr/share/perl5/vendor_perl/esmith/
. For instance,
NethServer::Password
Secret generation and persistent storage, under/var/lib/nethserver/secrets/
.
- NethServer::Service
- Service manager agnostic API. No matter if a service is managed by systemd, Upstart or SysV init script: use this API to gain control over it.
- NethServer::Directory
- Access to LDAP, service accounts and ACL management, low-level user and group management.
- NethServer::MailServer
- Obtain accounts and mail addresses relations. Manage IMAP ACLs.
- esmith::templates
- Template processing and expansion.
- esmith::events
- Event execution and tracking.
For more informations about a specific module, refer its perldoc documentation.
Events¶
Events are a mechanism which allows the system to trigger a set of actions in response to actual events that happen on the system. When one of the users interfaces modifies the configuration databases, it must signal an event to regenerate the various server application configuration files according to the new configuration.
Note: The user interface must never modify configuration files directly. Neither should to the administrator from command line.
Each event is associated with a list of actions which should be
performed when the event occurs and is defined as a subdirectory of
/etc/e-smith/events/
containing symbolic links to the appropriate
actions, loosely modelled after the ‘’System V init’‘mechanism for
starting servers. For example, if you examine the
/etc/e-smith/events/interface-update
directory:
[root@nsrv actions]# ll /etc/e-smith/events/interface-update/
total 8
lrwxrwxrwx. 1 root root 34 Feb 6 11:19 S04interface-config-adjust -> ../actions/interface-config-adjust
lrwxrwxrwx. 1 root root 33 Feb 6 11:19 S25interface-config-reset -> ../actions/interface-config-reset
lrwxrwxrwx. 1 root root 33 Feb 6 11:19 S30interface-config-write -> ../actions/interface-config-write
lrwxrwxrwx. 1 root root 35 Feb 6 11:19 S70interface-config-restart -> ../actions/interface-config-restart
lrwxrwxrwx. 1 root root 36 Feb 6 11:19 S75interface-config-hostname -> ../actions/interface-config-hostname
drwxr-xr-x. 2 root root 4096 Feb 6 11:20 services2adjust
drwxr-xr-x. 3 root root 4096 Dec 18 11:17 templates2expand
The symbolic links are given prefixes such as S15, S85, etc. to specify
the order in which the actions should be executed in a similar manner to
the System V init mechanism.
You can change the actions performed by an event by changing the links
in the event directory. You can also create a new event by creating
another subdirectory of /etc/e-smith/events/
.
Implicit actions¶
Most events contain two common tasks: expanding various templates and
adjusting (e.g. restarting) the relevant services. For this reason, two
implicit actions are included in all events. These implicit actions mean
that additional code does not need to be written to perform these common
tasks. The implicit actions are represented by entries in the
services2adjust/
and templates2expand/
subdirectories.
services2adjust¶
The services2adjust/
directory contains links mapping a specific
service to the action to perform on that service. For example, if
signalling the event in question requires that the ntpd service is
restarted, you simply include the link ntpd -> restart in the
services2adjust
directory. The implicit action services2adjust would
then restart the ntpd service. As an example, the services2adjust/
directory for the nethserver-httpd-update
event is shown below:
# ls> l /etc/e-smith/events/nethserver-httpd-update/services2adjust/
total 0
lrwxrwxrwx. 1 root root 7 Oct 2 09:05 httpd -> restart
templates2expand¶
The templates2expand/
directory contains a list of the configuration
files which need to be regenerated from their templates. This list
consists of a collection of empty files with the same file name as the
configuration file to be expanded and in a heirarchy mirroring their
location on the system. For example, to expand templates for the
/etc/samba/smb.conf
configuration file, simply include the empty
file etc/samba/smb.conf
in the templates2expand/
directory of
the relevant event.
Order of implicit actions¶
The implicit actions are implemented by inserting the action script
generic_template_expand
early in the list of actions to be run in an
event and the adjust-services
action near the end of the list.
You should normally link your action scripts in the range S10 to S80 so
that they occur after templates2expand and before services2adjust.
Note
The generic_template_expand
action is currently run at
S05 and adjust-services
is run at S90.
Signalling events¶
The signal-event
program takes an event name as an argument, and
executes all of the actions in that event, providing the event name as
the first parameter and directing all output to the system log. It works
by listing the entries in the event directory and executing them in
sequence. So for example, the command:
signal-event interface-update
will perform all the actions associated with the interface-update
event, which is defined by the contents of the
/etc/e-smith/events/interface-update/
directory.
Events with arguments¶
So far we have described the following general principle throughout the system; changes are made by altering the database values, then signalling events. The actions triggered by each event typically regenerate entire configuration files, taking into account the latest configuration information.
However, some changes are best made incrementally. For example, consider
the user-create event. One of its actions updates the LDAP directory,
which it could do by deleting all of the users and recreating them based
on the updated accounts
database. However, this is inefficient and
would lose any additional LDAP attributes which may have been stored. It
would be better to simply add the new user incrementally, using the
default LDAP schema.
But how is the action code to know which user was just added? The new
username is passed as an argument to the user-create event. This way the
action programs triggered by the user-create event have a choice. They
can either ignore the username argument and regenerate their output
based on the updated list of accounts, or they can pay attention to the
username argument, retrieve the rest of the information about the new
user from the accounts
database, and perform the incremental work to
add the user.
Note
Action scripts should normally take at most two arguments. The first is always the event name. The second optional argument is a key into one of the databases.
Events are not currently serialized. In most cases overlapping events will not cause issues, but caution should be exercised when events are signalled from programs.
Standard events and their arguments¶
The table below summarises the key NethServer events and their argument if required. Remember, each action script is always called with the event name as the first argument. The arguments listed in this table are provided as the second argument.
Event | Arguments | Description |
---|---|---|
certificate-update | The server public key certificate has been updated | |
group-create | Group key | Called when a group is created |
group-delete | Group key | Called when a group is deleted |
group-modify | Group key | Called when a group is modified |
group-create-pseudonyms | Signalled when the automatic creation of group email address is required | |
host-create | Host key | Called when a host is created |
host-delete | Host key | Called when a host is deleted |
host-modify | Host key | Called when a host is modified |
hostname-modify | Called when the SystemName or DomainName keys have been modified | |
ibay-create | Shared folder key | Called when a shared folder is created |
ibay-delete | Shared folder key | Called when a shared folder is deleted |
ibay-modify | Shared folder key | Called when a shared folder is modified |
interface-update | Called when a network interface configuration is updated in networks db | |
logrotate-update | Change default log retention and rotation policies | |
trusted-networks-update | The set of trusted networks is changed | |
migration-import | Path to migration directory | Import migration data from the given directory |
notifications-save | Set notification configuration (root forward, mail sender address) | |
password-expired | Username, expire date | The given username password will expire on expiredate |
password-modify | User key | Called when a user password is modified |
password-policy-update | User key | Called when the system password policy has been changed |
post-backup-config | Called after configuration backup end | |
post-backup-data | Called after data backup end | |
post-restore-config | Called after restore of configuration | |
post-restore-data | Called after restore of data | |
pre-backup-config | The pre-backup-config event creates consistent system state for the backup | |
pre-backup-data | The pre-backup-data event creates consistent system state for the backup | |
pre-restore-config | Called before restore of configuration | |
pre-restore-data | Called before restore of data | |
pseudonym-create | Pseudonym key | Called when a pseudonym is created |
pseudonym-delete | Pseudonym key | Called when a pseudonym is deleted |
pseudonym-modify | Pseudonym key | Called when a pseudonym is modified |
user-create | User key | Called when a user is created |
user-cleanup | User key | Remove all user data |
user-delete | User key | Called when a user is deleted |
user-modify | User key | Called when a user is modified |
user-create-pseudonyms | User key | Called when the automatic creation of user’s email address(es) is required |
user-lock | User key | Called when a user account is locked |
user-unlock | User key | Called when a user account is unlocked |
system-initialization | Initialize all system after installation | |
software-repos-save | Configure software repositories |
Handling deletions¶
When adding a user, the user is created in the accounts
database,
and various actions, such as creating the Linux account, are performed
in the user-create
event. However, when deleting a user, we want to
maintain the accounts
database entry for as long as possible, in
case there is information which the actions in the user-delete
event
might need in order to cleanly delete the users.
The system convention for handling deletions is:
- Change the type of the entry to mark it as being in the process of being deleted e.g. a’‘user’‘entry becomes a’‘user-deleted’‘entry.
- Signal the relevant deletion event - e.g.’‘user-delete’‘
- Remove the entry from the database, but only if the event succeeds. With this approach, the action scripts can decide whether to ignore the’‘user-deleted’’ entries when performing their tasks.
Event logs¶
Warning
Output of event logs will be soon refactored!
All events, and all actions run by the event, are logged to the
messages
system log. Here is an example action log, which has been
formatted onto multiple lines to enhance readability:
Feb 2 13:22:33 gsxdev1 esmith::event[4525]:
S65sshd-conf=action|
Event|remoteaccess-update|
Action|S65sshd-conf|
Start|1138846952 730480|
End|1138846953 66768|
Elapsed|0.336288
From this single log, we can see the action script name, which event it was called in, when it started, ended and how long it took (0.34 seconds). Now, let’s add an action script which always fails and signal the event again:
Feb 2 16:11:54 gsxdev1 esmith::event[4787]:
S99false=action|
Event|remoteaccess-update|
Action|S99false|
Start|1138857114 58910|
End|1138857114 81920|
Elapsed|0.02301|
Status|256
Note that this log has a new field Status, which is added if the action script returns a false (non-zero) exit status. Suppressing the Status field when it is zero (success) makes it much easier to find failed actions in the logs.
Warning
If an action script fails, the entire event fails. The other actions scripts in the event are run, but the whole event is marked as having failed.
System validators¶
System validators provide an extensible UI-independent data validation layer.
On one hand UI implements fast grammar and/or syntax checks on input data. On the other, the system validators performs in-depth system consistency checks.
Design¶
Validators have a behaviour very similar to events.
- A validator is a directory inside
/etc/e-smith/validators
. - Each validator directory has a descriptive name, eg. user-name for a validator which validate a new user name.
- A validator is composed by an arbitrary number of actions saved inside
/etc/e-smith/validators/actions
directory and linked inside validator directory. - A success validation occurs when all scripts return 0 (success validation) or at least one script returns 2 (sufficient valid condition).
A validator action are always called with a single parameter which is the value to be validated. Actions must return one of these exit values:
- 0: successful validation
- 1: validation failed
- 2: sufficient validation
- other value: specific error state
When a script returns 2 (sufficient validation) no further script will be processed.
Inside nethserver-devtools package there is validator_actions()
function which help creating links to actions just like event_actions
function. See perldoc esmith::Build::CreateLinks
for details.
Invoking a validator:
validate <validator-name> <value-to-validate>
Eg:
validate user-name goofy
Services¶
A service is a software which usually runs in background.
The system will ensure service status accordingly to its configuration.
A service in configuration
database is something like this:
httpd=service
status=enabled
access=public
TCPPorts=80,443
Where httpd
is the service name and status
tells the system if the service should be enabled
or disabled
.
When the status property is switched between enabled/disabled state, the change will be reflected into runlevel configuration. This is what runlevel-adjust event and action do for all configured services. There is also another action called adjust-services which does the same thing for services registered on a single event.
A service without a record in the configuration database is ignored and can be manually manged using systemctl. See Add a new service.
Control a service¶
Enable a service:
config setprop myservice status enabled
signal-event runlevel-adjust
Disable a service:
config setprop myservice status disabled
signal-event runlevel-adjust
Where myservice
is the service name to be enabled or disabled.
Access network service¶
A network service is a service running on the server which expose UDP or TCP ports. Ports can be listed in following properties:
TCPPort
: a single TCP portTCPPorts
: a comma separated list of TCP portsUDPPort
: a single UDP portUDPPorts
: a comma separated list of UDP ports
If both TCPPort and TCPPorts properties are set, TCPPorts has the precedence. If both UDPPort and UDPPorts properties are set, UDPPorts has the precedence.
A service can be accessible from public or private LAN. This configuration is saved on access
property.
The property is a comma separated list of zones (green, red, blue, orange).
Example of a service with UDP port 1122 open to the Internet:
config setprop myservice status enabled UDPPort 1122 access green,red
Example of a service with TCP ports 1122 an 2233 open to local network:
config setprop myservice status enabled TCPPorts 1122,2233 access green
The ports are opened only if the status
property is set to enabled
.
Add a new service¶
Any software can configure the init system using the standard systemctl command. This approach always work for third-party software.
On the other hand, if the service must be controlled by NethServer, create a new record inside configuration database:
config set myservice service status enabled
Where myservice
is the name of the new service.
Make sure also there are defaults values inside the directory /etc/e-smith/db/configuration/defaults
: if the key is present
inside the configuration database, but not inside defaults, the service will be stopped.
Given the above example, create these files:
mkdir -p /etc/e-smith/db/configuration/defaults/myservice
echo "service" > /etc/e-smith/db/configuration/defaults/myservice/type
echo "enabled" > /etc/e-smith/db/configuration/defaults/myservice/status
Signal the new service to the system:
signal-event runlevel-adjust
Add a new network service¶
If a service not controlled by NethServer needs one or more open ports, use the TCPPort(s) or UDPPort(s) prop to declare the port(s) and signal the firewall to open it:
config set fw_myservice service status enabled TCPPort 12345 access green
signal-event firewall-adjust
Otherwise, if the service is controlled by NethServer, you can add the properties directly to the service key. For the service myservice on above example:
config set myservice service status enabled TCPPort 12345 access green
signal-event firewall-adjust
Auto-generated random URL¶
Sometimes you need to install web applications which don’t have built-in authentication. A good solution can be an auto-generated random URL known only to some special users. It’s also a best practice to restrict access to those applications using Apache allow and deny rules.
This feature is implemented in nethserver-lib using
genRandomHash
function. The function will generate a SHA1 random
hash
Example from [[nethserver-collectd-web]]:
my $alias = $collectd->prop('alias') || "";
# initialize alias if needed
if ($alias eq "") {
$alias = esmith::util::genRandomHash();
$confdb->set_prop('collectd-web','alias',$alias);
}
Random alias should be generated inside an action, like
<package_name>-conf
. The action must be executed before
template-expand in a position before 05.
Example from createlinks:
my $event = "nethserver-samba-audit-update";
event_actions($event, qw(
initialize-default-databases 00
nethserver-samba-audit-conf 02
));
Migration from NethService/SME Server¶
Migration is the process to convert a SME Server (or NethService) machine into a NethServer.
In the old host, create a full backup archive and move it to the new NethServer host.
In the new server, install all packages that cover the same features of the old one.
Explode the full backup archive on some directory (for instance
/var/lib/migration
)Signal the event:
signal-event migration-import /var/lib/migration
This step will require some time.
Search for any
ERROR
string in/var/log/messages
Coding conventions¶
Most modules have already a migration action which handles the step automatically.
A migration action:
- must be named as
<packagename>-migrate
- must be linked into
migration-import
event - must migrate old properties values to new ones
- can copy original data files to the new location
- must take care to apply the imported configuration, possibly using
the
<packagename>-update
event
During migration some properties will not be imported:
- UDPPort, TCPPort, UDPPorts, TCPPorts: all network ports will be reset to new defaults
- DNS forwarder, green IP address, default gateway: these properties are filled up in bootstrap-console
All e-smith databases are moved in /var/lib/nethserver/db
directory.
Code snippets¶
A simple migrate action in perl.
#!/usr/bin/perl
use esmith::DB::db;
use esmith::event;
use strict;
my $event = shift;
my $sourceDir = shift;
my $esmithDbDir = 'home/e-smith/db';
my $errors = 0;
if {
die;
}
my $srcConfigDb = esmith::DB::db]>open\_ro(join('', $sourceDir, $esmithDbDir,'configuration')) || die("Could not open source configuration database in $sourceDir");
my $dstConfigDb = esmith::DB::db->open || die;
my $service = ‘ejabberd’;
my $old = $srcConfigDb->get($service);
my $new = $dstConfigDb->get || $dstConfigDb->new_record($service);
$new->merge_props($old->props);
# Apply configuration
if( ! esmith::event::event_signal('nethserver-ejabberd-update')) {
exit(1);
}
exit 0;
Remember to change the service name and add a license header.
Add the migrate action to createlinks:
#-----------------------------------
# actions for migration-import event
#-----------------------------------
$event ="migration-import";
event_actions($event, '<packagename>-migrate' => 60);
Packages¶
Each packages with special migration notes is listed below.
nethserver-base¶
Properties not migrated:
- PasswordSet
- UnsavedChanges
- bootstrap-console
- dns
- sysconfig
- SystemMode
- green network configuration
- ActiveAccounts (moved to nethserver-directory, calculated on-the-fly)
nethserver-backup¶
No migration is possible. The backup must be reconfigured.
nethserver-directory¶
Home directories: user’s home directoriy migrates into /var/lib/nethserver/home
,
admin’s home directory migrates into /var/lib/nethserver/migration/admin
, and a symlink is created in /root/admin-migration-<TIMESTAMP>
.
nethserver-hylafax¶
After migration check the configuration of incoming fax notification.
nethserver-httpd¶
The ibay-virtualhost relation has been designed differently from SME/NethService. An automatic migration is not always possible; the resulting configuration must be checked manually.
The global-pw-remote’ case is currently not implemented in NethServer and is mapped as global-pw
.
The reason is we do not want make distinctions between internal/external connections.
nethserver-mail-server¶
During pseudonyms migration,
- pseudonyms pointing to
admin
andshared
accounts are mapped topostmaster
, as any other account not existing in destination AccountsDB. Thus the resulting configuration requires post-migration supervision. - recursive pseudonyms (pointing to another pseudonym) are flattened and a relation with a user or group account record is established.
Index of shared mailboxes is not migrated. Each user must re-share its own mail directory.
To workaround this problem copy the original index file (/etc/dovecot/sharedmailbox/dict.db
) to the new location (/var/lib/nethserver/vmail/shared-mailboxes.db
) and restart dovecot.
See http://wiki2.dovecot.org/SharedMailboxes/Shared for more information.
Forbidden “" in folder names¶
The dovecot plugin listplugin (http://wiki2.dovecot.org/Plugins/Listescape) is enabled, and uses backslash “" as escape character. If original folder names contains “", run the following command after post-migration mail synchronization, to rename them:
find /var/lib/nethserver/vmail/ -type d -regex '.*\\.*' -prune | (while read -r SRC; do echo mv -iv "$SRC" "${SRC//\\/\\5c}"; done )
nethserver-mail-filter¶
No wildcards expansions are supported by nethserver-mail-filter UI interface; only full mail addresses or domain names. The migration action must map email addresses in the form *domain.tld
to domain names, and log a warning whenever another form of wildcard expansion is used.
Also recipient blacklists are not implemented and bayes DB is not migrated
Certificate Management¶
nethserver-base provides a set of templates that output PEM-formatted certificate parts:
- certificate/key RSA private key
- certificate/crt public certificate
- certificate/pem both key+crt parts
Configuration is inside the configuration
database. Example:
pki=configuration
KeyFile=
CrtFile=
ChainFile=
CertificateDuration=365
CommonName=
A certificate consumer daemon should expand those templates to its own
certificate paths, by installing the proper configuration under
/etc/e-smith/templates.metadata
.
For instance nethserver-httpd adds the following template configuration:
/etc/e-smith/templates.metadata/etc/pki/tls/private/localhost.key
TEMPLATE_PATH="certificate/key"
OUTPUT_FILENAME="/etc/pki/tls/private/localhost.key"
PERMS=0600
UID="root"
GID="root"
/etc/e-smith/templates.metadata/etc/pki/tls/certs/localhost.crt
TEMPLATE_PATH="certificate/crt"
OUTPUT_FILENAME="/etc/pki/tls/certs/localhost.crt"
PERMS=0600
UID="root"
GID="root"
Set OUTPUT_FILENAME
, PERMS
, UID
and GID
values according
to daemon configuration.
Default behavior¶
By default, CrtFile
and KeyFile
properties have empty values. In
this case, nethserver-base
generates a self-signed certificate
during nethserver-base-update
event.
Default SELinux-aware certificate locations are:
/etc/pki/tls/private/NSRV.key
: private key/etc/pki/tls/certs/NSRV.crt
: CA certificate
A daily cron job checks certificate validity. If expired, the
self-signed certificate is re-generated and certificate-update
event
is signaled.
Default certificate duration is set to 365 days. To change it:
db configuration setprop pki CertificateDuration 3650
The certificate Common Name is set to system FQDN. To override this value type:
db configuration setprop pki CommonName custom.cn
Let’s Encrypt¶
NethServer can request and renews Let’s Encrypt (LE) certificates.
The main helper /usr/libexec/nethserver/letsencrypt-certs
can be executed also from command line.
For more info, see:
/usr/libexec/nethserver/letsencrypt-certs -h
Database properties under pki
key inside configuration
database:
LetsEncryptMail
: (optional) registration mail for LE notificationsLetsEncryptDomains
: comma-separated list of domains added to certificate SAN fieldLetsEncryptChallenge
: challenge to use for validating the certificate, default ishttp
. It accepts also values likedns-<provider>
. Where<provider>
is the name of the DNS provider. See the full list of available DNS provider plugins by executingcertbot -h certonly
. More info at https://certbot.eff.org/docs/using.html?highlight=dns#dns-plugins.
DNS challenge¶
To use the DNS challenge, follow these steps:
- install the required certbot plugin plugin using yum; to see the list of available package use
yum search certbot-dns
- set
LetsEncryptChallenge
property to correct DNS plugin - configure all required properties accordingly to plugin documentation
When using the dns challenge, make sure to set extra properties accordingly to certbot configuration.
All properties for the dns challenge should be in the form LetsEncrypt_<certbot_option>
, where
<certbot_option>
is the option specific to certbot DNS plugin.
Digitial Ocean example¶
Install the plugin:
yum install python2-certbot-dns-digitalocean
Configure the challenge type:
config setprop pki LetsEncryptChallenge dns-digitalocean
Configure required props accordingly to https://certbot-dns-digitalocean.readthedocs.io/en/stable/:
config setprop pki LetsEncrypt_dns_digitalocean_token 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
Request certificate for domain
myserver.nethserver.org
:/usr/libexec/nethserver/letsencrypt-certs -v -d myserver.nethserver.org
Yum plugin¶
The nethserver-yum
package contains the nethserver_events
which extends the post-RPM transaction hook. It executes the *-update
event of each nethserver-*
package involved in the transaction, preserving the order of RPM dependencies.
Signalling update
events after RPM cleaning up assures that any old event handler and template fragments have been removed.
The configuration file is /etc/yum/pluginconf.d/nethserver.conf
. Available options are:
- enabled: 1 (default) or 0, enable or disable the plugin
- verbose: 1 or 0 (default), enable or disable debugging messages
Caching¶
When NethServer is behind a proxy server you can force to bypass an intermediate caching by adding this option to the yum.conf
file, under the main
section:
http_caching=none
Be aware that yum.conf is a template and not using the cache can raise load on remote repository servers, use with care!
Web interface¶
The web interface is the main configuration method for NethServer. The main goal is to hide the system configuration complexity behind a simple and clear interface.
It runs on a special httpd instance and is accessible at: https://your_server:980
or https://your_server/server-manager
(if nethserver-httpd module is installed).
Nethgui framework¶
Nethgui framework is provided with a set of components and basic classes to quickly build a modern web user interface. Its main goals are:
- provide a simplified API for coding, localizing and realizing the web user interface and behaviour for project:NethServer based products.
- perform user authentication and authorization;
- offer test facilities to help module testing and debugging.
Key features:
- Object-oriented and embeddable
- Plugin support
- REST-aimed architecture
- Almost no external dependencies
- User roles
- AJAX driven HTML5 Widget library
- Unobtrusive JavaScript code
- Compatible with console-based and mobile browsers
NethServer Web User Interface is built upon Nethgui framework.

Creating web UI module¶
In this page you can find a working example of a simple UI for a new module, but, first a little of theory.
Web UI structure¶
All code is organized in three main directories:
/usr/share/nethesis/Nethgui
: framework libraries (can be used for other projects)/usr/share/nethesis/NethServer
: actual implementation of modules UI/usr/share/nethesis/nethserver-manager
: web root directory
Nethgui is a MVC framework that aims to abstract the developer from the backend and frontend. This means that you don’t have to deal with reading and writing properties from DB, neither care about HTML, CSS and JavaScript on the client side. You just write the controller. Of course, if you want, you can play with all three layers but it shouldn’t be necessary in the most of cases.
Each module is composed by 4 parts:
- Controller:
/usr/share/nethesis/NethServer/Module/.php
- View:
/usr/share/nethesis/NethServer/Template/.php
- Translation:
/usr/share/nethesis/NethServer/Language//NethServer\_Module*.php
- Inline help :
/usr/share/nethesis/NethServer/Help/NethServer_Module*\ .html
If needed, a module can add extra resources:
- Specific authorization (JSON format):
/usr/share/nethesis/NethServer/Authorization/.json
- See Nethgui:Authorization - Custom CSS:
/usr/share/nethesis/NethServer/Css/.css
- See Including CSS Custom JavaScript: /usr/share/nethesis/NethServer/Js/.js
- See Including JS- Unit tests:
/usr/share/nethesis/NethServer/Test/Unit/NethServer/Module/.php
- See Nethgui unit test - Utility libraries:
/usr/share/nethesis/NethServer/Tool/.php
Writing the code¶
We add a UI to the package nethserver-ejabberd module. The UI will
expose 2 properties of the ejabberd
db key.
Properties:
- status: start and stop the ejabberd daemon on boot, can be enabled or disabled
- WelcomeText: welcome text, can be anything
Controller¶
First, we create the controller which has 3 main functions:
- initializeAttributes: handle module position in menu
- initialize: bind the properties to the database and set the validator
- onParametersSaved: apply the configuration
Here is the controller
(/usr/share/nethesis/NethServer/Module/Ejabber.php
):
class Ejabber extends \Nethgui\Controller\AbstractController
{
// Add the module under the 'Configuration' section,
protected function initializeAttributes(\Nethgui\Module\ModuleAttributesInterface $base){
return \Nethgui\Module\SimpleModuleAttributesProvider::extendModuleAttributes($base, 'Configuration', 30);
}
// Declare all parameters
public function initialize(){
parent::initialize();
// Bind 'WelcomeText' view parameter to 'WelcomeText' prop in ejabberd key of configuration db
$this->declareParameter('WelcomeText', Validate::ANYTHING, array('configuration', 'ejabberd', 'WelcomeText'));
}
// Execute actions when saving parameters
protected function onParametersSaved($changes) {
// Signal nethserver-ejabberd-save event after saving props to db
$this->getPlatform()->signalEvent('nethserver-ejabberd-save@post-process');
}
}
View¶
Show all fields using built-in functions. If needed, you can add extra HTML markup but remember that the output must be functional on any device (desktop, mobile, text browser, etc).
Template (/usr/share/nethesis/NethServer/Template/Ejabber.php):
echo $view->header()->setAttribute('template', $T('Ejabber_Title'));
// add simple panel
echo $view->panel()
//add 'status' parameter checkbox with value when checked and unchecked
->insert($view->checkbox('status', 'enabled')->setAttribute('uncheckedValue', 'disabled'));
//add 'WelcomeText' text input field
echo $view->panel()->insert($view->textInput('WelcomeText'));
;
// show submit and help buttons
echo $view->buttonList($view::BUTTON_SUBMIT | $view::BUTTON_HELP);
Translation¶
Translation files, are simple PHP files containing an associative
array.
All module language files are placed in /usr/share/nethesis/NethServer/Language/<lang>
.
Given a module with name “Test”, the english language file will be /usr/share/nethesis/NethServer/Language/en/NethServer_Module_Test.php
.
Warning messages about missing translations can be found in /var/log/messages
after Nethgui debug is enabled.
To enable the debug, use index_dev.php on urls, eg: https://<ipaddress>/index_dev.php/en/<module>
.
English translation (/usr/share/nethesis/NethServer/Language/en/NethServer_Module_Ejabber.php):
<?php
$L['Ejabber_Title'] = 'Chat server';
$L['status_label'] = 'Enable Ejabber chat server';
$L['WelcomeText'] = 'Welcome!';
Inline help¶
Help pages are RST documents compiled into xHTML pages at package build time.
===========
Chat server
===========
Ejabber is a chat server that implements the Jabber/XMPP protocol Jabber / XMPP, it support TLS on standard XMPP ports (5222 or 5223).
The chat server uses system users to login.
More examples¶
More examples can be found here or browsing the existing modules.
Dashboard¶
The dashboard module is the landing page of NethServer Web UI. It aims to give an overview of system status. Dashboard is fully pluggable and extensible: each NethServer module can add it’s own tab or even a widget inside the System status or Applications tab.
There are three default tabs:
- System status
- Services
- Applications
The dashboard module heavily uses Javascript but works correctly even on smartphones.
SystemStatus tab¶
It’s the main tab composed by three main submodules:
- System release: show the system release
- Resource usage:
- system load
- cpu number
- uptime
- physical memory usage statistics
- virtual memory (swap) usage statistics
- root partition usage statistics
- Network
- configuration of each assigned netwok interface (IP address, netmask, etc.)
- real-time statistics of received and transmitted data
- host name, domain name, default gateway
This tab can be extended using plugins.
Services tab¶
Service tab lists all system services, including current status (running or stopped), configured status (enabled or disabled) and associated TCP and UDP ports. This tab cannot be customized.
Applications tab¶
Applications tab lists all installed application with extra information. Each package can create a special PHP class which describes the installed application.
Extending the dashboard¶
The dashboard heavily uses Nethgui framework to render all information, so you need a little bit of understanding on how project:Nethgui PHP framework actually works. Don’t panic!
Nethgui is a MVC framework specially designed for NethServer web UI tasks.
All files are inside /usr/share/nethesis/NethServer
path, with the following sub-paths:
- Module: controller files
- Template: template files
- Language: language files
Application custom widget¶
This is the simplest widget inside the Applications tab. This kind of widget should be used by all web applications.
Let see the nethserver-collectd-web example.
The package must include a PHP file inside Module/Dashboard/Applications/
directory. The script will describe a new class which extends the AbstractModule class and implements the ApplicationInterface interface. A little scary, eh? There is the full example:
<?php
class Collectd extends \Nethgui\Module\AbstractModule implements \NethServer\Module\Dashboard\Interfaces\ApplicationInterface
{
public function getName()
{
return "Collectd Web";
}
public function getInfo()
{
$cweb = $this->getPlatform()->getDatabase('configuration')->getKey('collectd-web');
$hostname = $this->getPlatform()->getDatabase('configuration')->getType('SystemName');
$domain = $this->getPlatform()->getDatabase('configuration')->getType('DomainName');
return array(
'url' => "http://$hostname.$domain/".$cweb['alias']
);
}
}
Not so scary :)
Basically, the ApplicationInterface requires to implement two simple methods:
- getName(): return the name of the application. The name is used to alphabetically sort all the applications inside the Application tab.
- getInfo(): return an arbitrary associative array. All <key,value> pairs will be printed inside a little auto-generated widget. If the array contains a key starting with ‘url’ string, the value of the key will be wrapped inside an
<a href=''>
HTML tag.
We can now analyze the above getInfo
function implementation. First we read the collectd-web
key from the configuration
db. All properties are then accessible as an associative array. Then we read the value of the special keys SystemName
and DomainName
. Finally we fill the return array with the url where the web application will be accessible.
System status custom widget¶
Let’s take a SystemRelease class which add a widget inside the SystemStatus tab:
- Controller:
Module/Dashboard/SystemStatus/SystemRelease.php
- Template:
Template/Dashboard/SystemStatus/SystemRelease.php
- English translation:
Language/en/NethServer_Module_Dashboard_SystemStatus_SystemRelease.php
- Italian translation:
Language/it/NethServer_Module_Dashboard_SystemStatus_SystemRelease.php
Translation files are simple PHP associative arrays. The language files in the examples are self-explanatory.
You can find all examples inside the nethserver-ui-examples repository.
Controller¶
First of all, let’s introduce few keys concepts about HTTP request handling in Nethgui.
When the browser open an URL, for example /SystemRelease
, the system will search for a module named SystemRelease. If the request contains some query parameters, the server side module will invoke two functions process() and prepareView(), if no query is specified only the function prepareView will be invoked.
The process will handle the query and prepare all data for the prepareView function. For example, here is the process implementation of a simple SystemRelease module:
<?php
...
public function process()
{
$this->release = $this->readRelease();
}
Simple. The process will invoke a private method readRelease which reads a file, and save the result on a private attribute. All data are now available and we have to send them back using the prepareView function:
<?php
...
public function prepareView(\Nethgui\View\ViewInterface $view)
{
//if no query is specified, make sure to initialize the data
if (!$this->release) {
$this->release = $this->readRelease();
}
$view['release'] = $this->release;
}
The release attribute is mapped inside the $view array, ready to be sent to the client. When the client requests the page for the first time with no query, we need to fill the release attribute because the process is not previously called. Then all the $view data will be used to fill the HTML template. If the client send a JSON request with a timestamp as a parameter (which is the standard behavior for ajax calls) the module will invoke process and prepareView, then all data will be formatted in JSON format.
Request examples:
- HTML rendering:
/Dashboard/SystemStatus/SystemRelease
- JSON response:
/Dashboard/SystemStatus/SystemRelease.json
Finally, if you want to order the widget inside the System Status tab, you should define a variable $sortId, like:
public $sortId = 40;
View¶
It’s time to create a simple template to show the data from the controller. Below there’s an example using a built-in CSS class.
<?php
echo "<div class='dashboard-item'>";
echo "<dl>";
echo $view->header()->setAttribute('template',$T('release_title'));
echo "<dt>".$T('release_label')."</dt><dd>"; echo $view->textLabel('release'); echo "</dd>";
echo "</dl>";
echo "</div>";
This is a static template without any use of Javascript. Inside the template file, you always have access to the $view variable where all data are stored by the previous explained prepareView function. There also a very useful function called $T(…) used for translations. The most important part of this examples is the call $view->textLabel(‘release’). This line is an helper which extract the release variable from the view an wrap it a span HTML tag identified by an auto-generated class (useful for Javascript processing).
A full example can be found in the nethserver-ui-examples repository under the dashboard directory.
Custom Tab¶
Any NethServer package can add a custom tab inside the dashboard. To create a new tab you have to write a class extending an existing controller inside the /usr/share/nethesis/NethServer/Module/Dashboard
directory.
See “API”:http://dev.nethserver.org/nethgui/Documentation/Api/ for a list of available controllers.
See mytab example in nethserver-ui-examples repository for more information.
Creating inline help pages¶
The online documentation is written in RST format. Each document is converted to XHTML at package build time. After installing a package, the server-manager web application serves the corresponding XHTML version.
To get a list of help pages visit
http://<your_server>:980/<lang>/Help
. The page will show a list of
available documents on the left column and a list of templates to
generate a new help page on the right column. You need to create an help
page for each supported language using the template: just copy and paste
the code to the destination file.
All help pages must be saved to the following absolute path:
/usr/share/nethesis/NethServer/Help/NethServer_Module_<name>.rst
Editing rules¶
To write an online help document, respect the following rules.
- The document must start with a title such as
==============
Document title
==============
- If a page is divided in tab, each tab title is in the form:
Tab title
=========
- Each field must be described as a definition list. The indentation of
My field
This is my description
- A field description can contain a bullet list:<pre>
My field
This is my description
* First element
* Second element
- If any words needs emphasis, use the asterisk character:
This is my *important* word.
# Actions, usually identified by buttons like Create or Configure should be given a separate section. Where two actions have similar forms, like Create and Modify, they can be merged together.
Create / Modify
---------------
- The Delete action should be documented only to clarify its side
- effects.
Creating help for plugin modules¶
Some NethServer modules have a plugin behavior which means the module will load all help documents inside a well known directory. Loaded files will be rendered as parts of the parent document.
Inside the parent document, add an include directive like this:
.. raw:: html
{{{INCLUDE NethServer_Module_User_Plugin_*.html}}}
When creating inline help for web ui plugin, the following rules apply:
# do not repeat the name of parent module
# use the ^
character for headers
# add this comment at the top of the file
.. --initial-header-level=3
The actual level value depends on the type of plugin. Usually 2 or 3 applies.
For instance, see NethServer_Module_SharedFolder_Plugin_Samba.rst
inside nethserver-samba package.
Building docs inside RPMS¶
RST documentation needs to be compiled in HTML. To include HTML files inside the RPM, remember to add this macro in the spec file, under the %build section:
%{makedocs}
Example:
%build
...
%{makedocs}
...
perl createlinks
RST editors¶
Here some RST editor with syntax hilighting.
Windows platform:
- http://notepad-plus-plus.org/
- www.geany.org
- http://www.sublimetext.com/ (non-free)
Linux platform:
- vim (of course!)
- gedit
Web:
TODO API¶
The TODO API is specific for the Server Manager web application. It is designed to execute a list of checks and possibly report the outcome to the admin user.
An RPM can install one or more executable scripts under
/etc/nethserver/todos.d/
.
The script must print the results formatted according to JSON [1] and the following schema:
{ "text": "free text", "icon": "info-circle", "action": { "url": "/User", "label": "Link label" } }
icon
andaction
keys are optional. The only required key istext
. Theurl
value is actually an absolute path to the Server Manager module. Future versions may support real URLs.If the script exit code is non-zero, or if the output is not correctly JSON-encoded, an error message is sent to the system log.
The script must be aware of locale settings, as its output is displayed on the user’s browser [2].
The executable helper /usr/libexec/nethserver/admin-todos
is
responsible for the invocation of scripts, validation of output and
error reporting. It is executed by the AdminTodo
UI module.
The AdminTodo
known callers are:
- Dashboard
- Software center
- Network
- Backup (configuration)
Translations¶
A TODO script must be locale-aware. Use the gettext
library for
code internationalization (i18n) and follow guidelines from
Internationalization.
References
[1] | JSON (JavaScript Object Notation) is a lightweight data-interchange format. http://json.org/ |
[2] | GNU gettext utilities http://www.gnu.org/software/gettext/ |
Building RPMs¶
To build RPMs for NethServer the following methods are provided:
- nethserver-makerpms for local builds with Podman [1] and GitHub Actions automated builds
- nethserver-mock local build with the Mock [2] configuration files pointing to NethServer YUM repositories
Both methods allow to build also RPMs from other projects. Check out the documentation carefully.
Development vs Release builds¶
A common convention about .spec
files says that the Release
tag has to append
the RPM dist
macro as suffix in a similar way:
Release: 1%{?dist}
In NethServer,
- when the package is built for a release, use the
dist
value.ns7
(or.ns6
…); - when the package is compiled for QA/testing, the
dist
macro should reference the source code git commit hash. For instance,dist
can be set to.2.g8f7ddad.ns7
.
This convention ensures that each RPM is unique and it is always possible to reproduce the build and its history.
In particular, when building for QA/testing, the build methods linked above prepend the
number of commits in the git log history since the last tagged release commit,
followed by .g
, followed by a short git commit hash reference.
If the last commit in the repo has a release tag, the dist
macro is set to .ns7
instead.
A tagged release commit is a git commit with a tag, starting with a digit
and not containing any “-” (minus) symbol.
For instance 0.1.2r1
and 0ok
are valid release tags, whilst v0.1.2
and 0.1.2-1
are not.
The releasetag command helps to create a properly tagged release commit in the RPM source code repository.
Publishing RPMs¶
Once RPMs are built, they can be uploaded to packages.nethserver.org
with sftp
.
Ask on community.nethserver.org to obtain the SFTP account required for this purpose.
Use the uploadrpms command to publish the RPMs in the current working directory. The command looks like the following
$ uploadrpms username@packages.nethserver.org:nscom/7.8.2003/nethforge-testing *.rpm
Replace 7.8.2003
with the correct NS version number. Also replace nethforge-testing
with the target repository name.
As alternative to the above manual procedure, it is possible to run the build on GitHub Actions:
it automatically publishes the RPMs to packages.nethserver.org
, too. Refer to
nethserver-makerpms documentation.
References
[1] | Podman is a daemonless Linux container engine. https://podman.io/ |
[2] | Mock is a tool for building packages. http://fedoraproject.org/wiki/Projects/Mock |
Building ISO¶
To create a NethServer ISO on a NethServer system, follow these steps:
Install
nethserver-createiso
packageMake sure mock cache is clean, execute as
root
user:rm -rf /var/cache/mock/nethserver-iso*
Log in as a non-privileged user, member of the
mock
groupDownload CentOS minimal ISO
Run
createiso
commandcreateiso -n nethserver -v 7.9.2009 -i CentOS-7-x86_64-Minimal-2009.iso
Package groups¶
The Software center page in the Server Manager displays the YUM groups (comps) metadata. This is an XML file added to a YUM repository metadata which describes the packages groups and categories composition.
Submit a change proposal¶
To modify how groups and categories are displayed in the Software center page a GitHub account is required.
- edit (on the GitHub web interface) an
.xml.in
file
- open a pull request
How to edit .xml.in
files¶
Information about *.xml.in
files is available here:
- about the workflow, see the repository README
- about the file format and the comps XML structure, see the Fedora Project Wiki
nethserver-antivirus¶
Manage default options for all clamav instances.
The configuration is saved inside the configuration
database in the clamd
key.
Properties:
OfficialSignatures
: can beenabled
ordisabled
. If set todisabled
, all default signatures will be deleted and freshclam will be disabled.UnofficialSignaturesRating
: can below
,medium
orhigh
. Select the rating risk of all sources. An higher rating means more virus detected but also an increased probability of false positives.
Database example:
clamd=configuration
OfficialSignatures=disabled
UnofficialSignaturesRating=low
Disable official signatures¶
Official signatures find only very few virus but have a big memory usage. To disable the official signatures:
config setprop clamd OfficialSignatures disabled
signal-event nethserver-antivirus-update
Select signature rating¶
Unofficial signatures can be selected using their rating. To change the rating use:
config setprop clamd UnofficialSignaturesRating medium
signal-event nethserver-antivirus-update
Cockpit API¶
dashboard/read¶
This api returns the information to show in dashboard page. The following data is retrieved for every ClamAV instance installed:
- current status of the instance
- number of malware found
- number of signatures loaded
- memory used by the ClamAV instance
- number of occurrences of malware by type
Input¶
- no input
Output example¶
{
"rspamd": {
"installed": true,
"active": true,
"malwareFound": 0,
"signatures": 172557,
"malwareStats": [],
"memoryUsedKb": 250808,
"sinceIso": "2019-09-23T12:19:01+0200"
},
"squidclamav": {
"installed": true,
"active": false,
"malwareFound": 0,
"signatures": 172557,
"malwareStats": [],
"memoryUsedKb": 0,
"sinceIso": "2019-09-23T12:18:04+0200"
}
}
logs/read¶
This api returns the list of installed ClamAV instances
Input¶
- no input
Output example¶
{
"installedInstances": [
"clamd@rspamd",
"clamd@squidclamav"
]
}
settings/read¶
This api retrieves clamd
configuration and the memory used by every ClamAV instance installed.
Input¶
appInfo
:clamdConfig
ortotalMemory
Input example (clamdConfig)¶
{
"appInfo": "clamdConfig"
}
Output example (clamdConfig)¶
{
"configuration": {
"type": "configuration",
"name": "clamd",
"props": {
"OfficialSignatures": "enabled",
"UnofficialSignaturesRating": "low"
}
}
}
Input example (totalMemory)¶
{
"appInfo": "totalMemory"
}
Output example (totalMemory)¶
{
"totalMemory": "991"
}
settings/validate¶
This api validates input for clamd
configuration.
Input¶
OfficialSignatures
:enabled
if ClamAV official signatures should be enabled, elsedisabled
UnofficialSignaturesRating
: Third-party signatures rating, valid values arelow
,medium
orhigh
. Higher rating means more blocked threats but also a more possibility of false positive. Recommended ratings arelow
andmedium
Input example¶
{
"OfficialSignatures": "disabled",
"UnofficialSignaturesRating": "low"
}
Output example¶
{
"state": "success"
}
nethserver-avahi¶
Configure avahi-daemon to start at boot. The Avahi mDNS/DNS-SD daemon implements Apple’s Zeroconf architecture.
It’s used by CUPS for printers auto-discovery. See cups-section.
Database:
avahi-daemon=service
UDPPort=5353
access=private
status=enabled
nethserver-backup-config¶
Configuration backup contains only system configuration files (passwd, config databases, etc). It’s scheduled to be executed every night and will create a new archive only if any file is changed in the last 24 hours.
Backup libraries use conf.d directory behavior (see perldoc NethServer::Backup
).
When a backup is started, the system will search for all files in /etc/backup-config.d
directory.
This directory can contain .include and .exclude files. Each file contain a list of file to include/exclude into/from the backup.
Example file /etc/backup-config.d/nethserver-base.include
/etc/e-smith/templates-custom
/etc/e-smith/templates-user-custom
/etc/ssh
/etc/sudoers
/etc/passwd
/etc/shadow
/etc/group
/etc/gshadow
Exclusions are evaluated after all inclusions.
All libraries are inside the nethserver-backup-config
package.
Configuration backup¶
The nethserver-backup-config
package implements the backup of configuration and relies on the backup-config
key inside the configuration
database.
Properties:
status
: enable or disable the automatic backup, can beenabled
ordisabled
. Default isenabled
.
Backup¶
The main command is /sbin/e-smith/backup-config
which starts the backup process (if enabled). The backup process has 3 steps:
- pre-backup-config event: used to prepare data, for example a LDAP dump of users
- backup-config-execute action: actually execute the backup if any file is changed in the last 24 hours.
The backup file is saved in
/var/lib/nethserver/backup/backup-config.tar.xz
(seeperldoc NethServer::BackupConfig
) - post-backup-config event: used to post-process the backup file, for example to copy the backup to a remote server or encrypting the archive
The configuration backup runs every night and it creates a new backup only if:
- destination file does not exist
- or new files are added or removed to/from the backup set
- or content of any file inside the set is changed
This package does not provide any default action in the pre-backup-config and post-backup-config events. But you can create a script inside the post-backup-config event to copy the configuration backup to a remote machine using, for example, the SSH protocol.
The configuration backup is included inside the data backup.
Restore¶
The main command is /sbin/e-smith/restore-config
which starts the restore process:
- pre-restore-config event: used to prepare the system, for example stop a running service
- restore-config-execute action: search for a backup file in the well-known directory (see above) and restore it
- post-restore-config event: used to apply restored configuration, for example reinstall packages and load the LDAP dump
This package does not provide any action in the pre-restore-config event.
Please note that if no /var/lib/nethserver/backup/backup-config.tar.xz
file is found, the restore-config
command
will try to access the data backup to retrieve the configuration backup.
The restore-config
options:
--no-reinstall
: disable package reinstall during restore--mask-unit=<name>
: use systemd to mask the specified unit (example: disable httpd-admin restart during restore)--skip-network
: do not restore the network configuration
Customization¶
Add custom include/exclude inside following files:
- /etc/backup-config.d/custom.include
- /etc/backup-config.d/custom.exclude
nethserver-backup-data¶
This module implements data backup for NethServer using different engines.
Available engines:
- duplicity: execute a full backup once a week, an incremental snapshot all other days of the week. Compression is enabled by default, encryption is not currently supported. It supports only storage backend which can be mounted on a local directory. Used also for the single backup.
- restic: always execute incremental backups using file deduplication. Encryption is always enabled, compression is not implemented. It supports local and remote backends.
- rsync: Time Machine-style backup using rsync. Very fast and reliable, the destination contains just regular files which can be easily accessed.
The nethserver-backup-data
package requires nethserver-backup-config
.
Configuration¶
Backups can be scheduled in different hours to multiple storage backends.
Global properties¶
This configuration is applied to all backups.
It uses the key backup-data
inside configuration
database.
Properties:
IncludeLogs
: if enabled, add/var/log
directory to backup, can beenabled
ordisabled
. Default isdisabled
.
Local properties¶
This configuration is applied only to the selected backups.
Every backup record is saved inside the backups
database. Each record can have 3 different types:
duplicity
restic
rsync
The key of the record is referred as the backup name
.
Common properties:
status
: enable or disable the automatic backup, can beenabled
ordisabled
. Default isenabled
. Regardless of this property, the backup is always executed if started manuallyBackupTime
: time of the scheduled backup. Must be in the cron-style syntax: Es.15 7 * * *
. Runs on 7:15.VFSType
: set the backup medium, can beusb
,cifs
,nfs
orwebdav
.SMBShare
: contains the Samba share nameSMBHost
: host name of the SMB serverSMBLogin
: login user for the SMB serverSMBPassword
: password for the SMB serverUSBLabel
: contains the filesystem labelNFSHost
: host name of the NFS serverNFShare
: contains the NFS share nameNotify
: if set toalways
, always send a notification with backup status; if set toerror
, send a notification only on error; if set tonever
, never send a notificationNotifyTo
: send the notification to given mail address, default isroot@localhost
WebDAVUrl
: contains the WebDAV URL addressWebDAVLogin
: login user for the WebDAV serverWebDAVPassword
: password for the WebDAV serverCleanupOlderThan
: time to retain backups, accept duplicity syntax (eg. 7D, 1M). Default is: never (keep all backups)
Supported VFSType:
cifs
: save the backup on a remote SMB server. Authentication is mandatory.nfs
: save the backup on a remote NFS server. No authentication supported.usb
: save the backup on a USB device. The device must have a writable filesystem with a custom label. When the backup is started, the system will search for an USB device with the filesystem label saved inUSBLabel
.webdav
: save the backup on a WebDAV server. When using a secure connection make sure the target WebDAV server has a valid SSL certificate, otherwise the system will fail mounting the filesystem.
Backward compatibility¶
To retain the backward compatibility with the old “single backup” feature, a backup named backup-data
has the following special features:
- can have a
NotifyFrom
prop to specify the sender address of notification mail - the backup can be modified from the old Server Manger and the status is reported inside the dashboard
- can be selectively restored using nethserver-restore-data package
Backup¶
The main command is /sbin/e-smith/backup-data -b <name>
which starts the backup process. The backup is composed of three parts:
- pre-backup-data event: prepare the system (eg. dump of mysql tables)
- /etc/e-smith/events/actions/backup-data-<program> action: execute the backup This actions must implement full/incremental logic and should also take care to mount and umount the destination
- post-backup-data: cleanup. Actions in this event can also implement retention policies
Logs and wrapper¶
Everything is logged to standard output and standard error.
If the backup is executed using backup-data-wrapper
,
a new log will be created inside /var/log/backup
.
After backup execution, the wrapper will also call
all executable scripts inside /etc/backup-data.hooks/
directory.
Each script is invoked with the following parameters:
- backup name
- log file
- backup exit code
Default hooks¶
backup-dashboard-status
: save the status of the backup/var/spool/backup/status-<backup_name>
backup-notify
: send backup notifications mails
Adding a backup¶
Create a backup record with all required options. Example: create a restic backup named
t1
using sFTP backend:db backups set t1 restic VFSType sftp SftpHost 192.168.1.123 SftpUser root SftpPort 22 SftpDirectory /mnt/t1 status enabled BackupTime 3:00 CleanupOlderThan 30D Notify error NotifyTo root@localhost
Enable the configuration:
echo -e "Nethesis,1234" > /tmp/t1-password; signal-event nethserver-backup-data-save t1 /tmp/t1-password
Start a backup¶
Start the backup, by passing the name of the backup to backup-data
command. Example:
backup-data -b t1
Disk usage¶
Each backup script creates statistics about disk utilization on the backup destination. Statistics are available only for: cifs, nfs and usb.
Data are saved inside /var/spool/backup/disk_usage-<backup_name>
.
Indexing¶
In the pre-backup-data event the disk analyzer (Duc) make an indexing of filesystem, useful to create the graphical tree.
The name of the actions is /etc/e-smith/events/actions/nethserver-restore-data-duc-index
and it compose the JSON file to create
the navigable graphic tree.
The indexing feature is limited to the backup named backup-data
.
Customization¶
Global¶
Add custom include/exclude inside following files:
/etc/backup-data.d/custom.include
/etc/backup-data.d/custom.exclude
This configuration is applied to all backups.
Local¶
Each backup can override the global list of include/exclude by creating two special files:
/etc/backup-data/<name>.include
/etc/backup-data/<name>.exclude
Where name
is the name of the backup.
Retention policy¶
All backups can be deleted after a certain amount of time. Cleanup process takes place in post-backup-data event.
See CleanupOlderThan
property.
Restore¶
Restore from command line¶
The main command is /sbin/e-smith/restore-data -b <name>
which starts the restore process:
- pre-restore-data event: used to prepare the system (Eg. mysql stop)
- restore-data-<program> action: search for a backup in the configuration database and restore it
- post-restore-data event: used to inform programs about new available data (eg. mysql restart)
To restore all data into the original location, use:
restore-data -b <name>
To restore a file or directory, use:
restore-file -b <name> <position> <path>
List backup contents¶
The list of file inside each backup can be obtained using:
/sbin/e-smith/backup-data-list -b <name>
Duplicity¶
The default program used for backup is duplicity using the globbing file list feature. Encryption is disabled and duplicity cache is stored in /var/lib/nethserver/backup/duplicity/ directory
.
We plan to support all duplicity features in the near future.
See http://duplicity.nongnu.org/ for more information.
Extra options¶
Properties valid only for duplicity engine, see “Single backup” section for an explanation of each property:
Type
: can befull
orincremental
. Iffull
, a full backup will be executed every time. Ifincremental
, a full backup will be executed once a week atFullDay
, all other backups will be incrementalFullDay
: number of day of the week when a full backup will be executed. Can be a number from 0 (Sunday) to 6 (Saturday). Defaults is0
.VolSize
: size of chunks in MB, if supported byProgram
. Default is 250
Listing backup sets¶
To list current backup sets:
- Mount the backup directory
- Query duplicity status
- Umount the backup directory
Just execute:
/etc/e-smith/events/actions/mount-`config getprop backup-data VFSType`
duplicity collection-status --no-encryption --archive-dir /var/lib/nethserver/backup/duplicity/ file:///mnt/backup/`config get SystemName`
/etc/e-smith/events/actions/umount-`config getprop backup-data VFSType`
Restic¶
Implement backup engine using restic (https://restic.net/), it can be used as duplicity replacement for standard backup or as multiple backup.
In restic, cleanup operations are composed by two commands: forget, to remove a snapshot, and prune, to actually remove the data that was referenced by the deleted snapshot. The prune operation is quite slow and should be executed at least once a week.
Extra options¶
Prune
: execute the pruning on the specified time. Valid values are:always
: run the prune every time at the end of backup- a number between
0
and6
: run the prune on the selected week day (0 is Sunday, 1 is Monday)
Storage backends¶
Supported VFSType
:
usb
cifs
nfs
webdav
s3
: Amazon S3 (or compatible server like Minio)sftp
: FTP over SSHb2
: BackBlaze B2rest
: Restic REST server
sftp¶
SFTP
Connection to remote host uses a specific public key. A password is needed only once to copy the public key to the remote host.
SSH client configuration is added to /etc/ssh/sshd_config
.
Properties:
SftpHost
: SSH host name or IP addressSftpUser
: SSH userSftpPort
: SSH portSftpDirectory
: destination directory, must be writable by SSH user
Example:
db backups set t1 restic status enabled BackupTime '15 7 * * *' CleanupOlderThan 30D Notify error NotifyTo root@localhost Prune 1 \
VFSType sftp SftpHost 192.168.1.2 SftpUser root SftpPort 22 SftpDirectory /mnt/t1
echo -e "Nethesis,1234" > /tmp/t1-password; signal-event nethserver-backup-data-save t1 /tmp/t1-password
The temporary file containing the password will be deleted at the end of nethserver-backup-data-save
event.
s3¶
Amazon S3 (https://aws.amazon.com/s3/) compatible (like https://www.minio.io/).
Properties
S3AccessKey
: user access keyS3Bucket
: bucket (directory) nameS3Host
: S3 host, use s3.amazonaws.com for AmazonS3SecretKey
: secret access key
Example:
db backups set t1 restic VFSType s3 BackupTime '15 7 * * *' CleanupOlderThan never Notify error NotifyTo root@localhost status enabled Prune always\
S3AccessKey XXXXXXXXXXXXXXXXXXXX S3Bucket restic-demo S3Host s3.amazonaws.com S3SecretKey xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx status enabled
signal-event nethserver-backup-data-save t1
How to setup Amazon S3 access keys: https://restic.readthedocs.io/en/stable/080_examples.html
b2¶
BackBlaze B2 (https://www.backblaze.com/b2/cloud-storage.html)
Properties:
B2AccountId
: B2 account nameB2AccountKey
: B2 account secret keyB2Bucket
: B2 bucket (directory)
Example:
db backups set t1 restic VFSType b2 BackupTime '15 7 * * *' CleanupOlderThan never Notify error NotifyTo root@localhost status enabled \
B2AccountId B2AccountId xxxxxxxxxxxx B2AccountKey xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx B2Bucket restic-demo
signal-event nethserver-backup-data-save t1
Rest¶
Restic REST server (https://github.com/restic/rest-server)
Properties:
RestDirectory
: destination directoryRestHost
: REST server host name or IP addressRestPort
: REST server port (default for server is 8000)RestProtocol
: REST protocol, can behttp
orhttps
RestUser
: user for authentication (optional)RestPassword
: password for authentication (optional)
Example:
db backups set t1 restic VFSType rest BackupTime '15 7 * * *' CleanupOlderThan never Notify error NotifyTo root@localhost status enabled \
RestDirectory t1 RestHost 192.168.1.2 RestPassword mypass RestPort 8000 RestProtocol http RestUser myuser
signal-event nethserver-backup-data-save t1
Database example:
t2=restic
BackupTime=1 7 * * *
CleanupOlderThan=never
Notify=error
NotifyTo=root@localhost
SMBHost=192.168.1.234
SMBLogin=test
SMBPassword=test
SMBShare=test
VFSType=cifs
status=enabled
t3=restic
BackupTime=15 7 * * *
CleanupOlderThan=never
NFSHost=192.168.1.234
NFSShare=/test
Notify=error
NotifyTo=root@localhost
VFSType=nfs
status=enabled
REST server¶
To manually install the REST server, download it from https://github.com/restic/rest-server/releases and save it
under /usr/local/bin/rest-server
, example Linux 64bit:
R=0.9.7; wget https://github.com/restic/rest-server/releases/download/v$R/rest-server-$R-linux-amd64.gz -O - | zcat > /usr/local/bin/rest-server
chmod a+x /usr/local/bin/rest-server
Then configure it for NethServer:
wget https://raw.githubusercontent.com/restic/rest-server/master/examples/systemd/rest-server.service -O - | sed 's/www\-data/apache/g' > /etc/systemd/system/rest-server.service
systemctl daemon-reload
systemctl start rest-server
systemctl enable rest-server
config set rest-server service TCPPort 8000 access green status enabled
signal-event firewall-adjust
rsync¶
Implement Time machine-style backup engine using rsync_tmbackup.sh
(https://github.com/laurent22/rsync-time-backup),
based on rsync (https://rsync.samba.org/). It can be used as duplicity replacement for standard
backup or as multiple backup.
Retention policy¶
Backup sets are automatically deleted when the disk is getting full.
More info on expiration strategy: https://github.com/laurent22/rsync-time-backup#backup-expiration-logic
Storage backends¶
Supported VFSType
:
usb
cifs
nfs
webdav
sftp
: FTP over SSH
sftp¶
SFTP
Connection to remote host uses a specific public key. A password is needed only once to copy the public key to the remote host.
SSH client configuration is added to /etc/ssh/sshd_config
.
Properties:
SftpHost
: SSH host name or IP addressSftpUser
: SSH userSftpPort
: SSH portSftpDirectory
: destination directory, must be writable by SSH user
Example:
db backups set t1 rsync status enabled BackupTime '15 7 * * *' Notify error NotifyTo root@localhost \
VFSType sftp SftpHost 192.168.1.2 SftpUser root SftpPort 22 SftpDirectory /mnt/t1
echo -e "Nethesis,1234" > /tmp/t1-password; signal-event nethserver-backup-data-save t1 /tmp/t1-password
The temporary file containing the password will be deleted at the end of nethserver-backup-data-save
event.
Database example:
t2=rsync
BackupTime=1 7 * * *
Notify=error
NotifyTo=root@localhost
SMBHost=192.168.1.234
SMBLogin=test
SMBPassword=test
SMBShare=test
VFSType=cifs
status=enabled
t3=rsync
BackupTime=15 7 * * *
NFSHost=192.168.1.234
NFSShare=/test
Notify=error
NotifyTo=root@localhost
VFSType=nfs
status=enabled
nethserver-base¶
This package implements all core configuration.
Network¶
Network configuration is saved inside the NetworksDB (/var/lib/nethserver/db/networks
).
Example of a database containing an interface:
eth0=ethernet
bootproto=none
device=eth0
gateway=192.168.1.254
ipaddr=192.168.1.1
netmask=255.255.255.0
onboot=yes
role=green
Each entry describes a network interface according to CentOS/RHEL specification for network-scripts files:
<device_name> = type
role = <role>
<param> = <value>
The type
variable is the type of interface. Valid values are:
- ethernet
- bond
- bridge
- alias
- ipsec
- xdsl
The <device_name>
variable is the name for the device.
The role
property is a mandatory parameter which describes the interface role. Valid values are:
- green
- orange
- blue
- red
If the role property is empty, the interface is not used by the system.
There are also 3 special roles:
- bridged: interface is part of a bridge
- slave: interface is part of a bond
- alias: interface is an alias of another interface
- xdsl-disabled: xdsl disabled interface
See also Roles and zones for the meaning of each color.
All <param>/<value>
are all valid CentOS network parameter for the specified interface. All parameters must be lowercase. Example:
- ippaddr
- dhcp_hostname
- netmask
- slave
- …
All parameters will be mapped 1-to-1 to the configuration file
Example
One green ethernet:
db networks set eth0 ethernet role green ipaddr 192.168.1.4 netmask 255.255.255.0 network 192.168.1.0 onboot yes bootproto static
File content:
green=ethernet|bootproto|static|device|green|ipaddr|192.168.1.4|netmask|255.255.255.0|network|192.168.1.0|onboot|yes|role|green
Bond options¶
Any property starting with BondOpt
prefix is used as bonding options.
Example:
bond0=bond
BondOptMode=0
BondOptMiimon=80
bootproto=none
gateway=192.168.1.100
ipaddr=192.168.1.2
netmask=255.255.255.0
role=green
Templates¶
The network database can be manipulated using the esmith::NetworksDB perl module. For more information use:
perldoc esmith::NetworksDB
If you need to access the local IP address within a template, use this code snippet:
use esmith::NetworksDB;
my $ndb = esmith::NetworksDB->open_ro() || return;;
my $LocalIP = $ndb->green()->prop('ipaddr') || '';
Note
Old templates used a variable called LocalIP to access the green IP address. This variable is no more available.
Events¶
All network configurations are applied by interface-update
event.
Database initialization¶
All interfaces are imported from configuration files to database using
the script: /usr/libexec/nethserver/update-networks-db
.
The networks database is updated Whenever an interface is plugged into the system.
DHCP on red interfaces¶
When configuring a red interface in DHCP mode, enable also the above options:
peer_dns
to avoid resolv.conf overwriting from dhclientpersistent_dhclient
to enforce dhclient to retry in case of lease request errors
Remember also to remove all gateway IP address from green devices. This configuration will create the correct routes and correctly set DHCP options on dnsmasq.
Bridge¶
Create a bridge interface from command line. The new interface will have green role (eth0 was the previous green interface):
db networks delprop eth0 ipaddr netmask bootproto
db networks setprop eth0 role bridged bridge br0
db networks set br0 bridge bootproto static device br0 ipaddr 192.168.1.254 netmask 255.255.255.0 onboot yes role green
signal-event interface-update
Reset network configuration¶
In case of misconfiguration, it’s possible to reset network configuration by following these steps.
Delete all logical and physical interfaces from the db
Display current configuration:
db networks show
Delete all interfaces:
db networks delete eth0
Repeat the operation for all interfaces including bridges, bonds and vlans.
Disable interfaces
Physical interfaces:
ifconfig eth0 down
In case of a bridge:
ifconfig br0 down brctl delbr br0
In case of a bond (eth0 is enslaved to bond0):
ifenslave -d bond0 eth0 rmmod bonding
Remove configuration files
Network configuration files are inside the
/etc/sysconfig/network-scripts/
directory in the form:/etc/sysconfig/network-scripts/ifcfg-<devicename>
. Where devicename is the name of the interface like eth0, br0, bond0.Delete the files:
rm -f /etc/sysconfig/network-scripts/ifcfg-eth0
Repeat the operation for all interfaces including bridges, bonds and vlans.
Restart the network
After restarting the network you should see only the loopback interface:
service network restart
Use ifconfig command to check the network status.
Manually reconfigure the network
Choose an IP to assign to an interface, for example 192.168.1.100:
ifconfig eth0 192.168.1.100
Then reconfigure the system:
signal-event system-init
The interface will have the chosen IP address.
Open the web interface and reconfigure accordingly to your needs
Zeroconf network¶
Zeroconf network (http://www.zeroconf.org/) shouldn’t be usefull on a server. It can be safely disabled using these commands:
config setprop sysconfig ZeroConf disabled
signal-event interface-update
Log retention and rotation¶
By default logs are rotated weekly and kept for 4 weeks. Some packages come with different defaults, but the majority do not specify a custom rotate value.
Logrotate db property:
Rotate
: rotation frequency, can bedaily
,weekly
,monthly
. Default isweekly
Times
: rotate log filesTimes
number of times (days, weeks or months) before being removes, default is 4Compression
: can beenabled
ordisabled
. Defaults isdisabled
Example:
logrotate=configuration
Compression=disabled
Rotate=weekly
Times=4
Keep logs for 6 months, rotate once a week:
config setprop logrotate Rotate weekly
config setprop logrotate Times 24
signal-event nethserver-base-update
Transport Layer Security¶
The TLS policy
page controls how individual services configure the
Transport Layer Security (TLS) protocol, by selecting a policy identifier.
Each module implementation decides how to implement a specific policy identifier, providing a trade off between security and client compatibility. Newer policies are biased towards security, whilst older ones provide better compatibility with old clients.
You can enforce the TLS policy (20180330), or choose the legacy one (empty policy property) if your clients are not supported/maintained anymore (Windows XP for example).
- TLS db property in configuration database: ::
- tls=configuration
- policy=
The event to expand the templates of all rpm which use TLS is tls-policy-save
Repositories¶
The default YUM repository set of NethServer is composed of
nethserver-base
: it contains packages and dependencies from core modules. It is updated when a new milestone is released. Enabled by default.nethserver-updates
: it contains updated packages. If needed, these updates can be applied without requiring manual intervention. Enabled by default.nethforge
: communty provided modules for NethServer. Enabled by default.nethserver-testing
: contains packages under QA process. Disabled by default.ce-base
: (ce-
stands for CentOS) base packages from CentOS. Enabled by default.ce-updates
: updated packages from CentOS. Enabled by default.ce-sclo-rh
andce-sclo-sclo
: SCL repositories. Both enabled by default.ce-extras
: extra RPMs. Enabled by default.epel
: Extra Packages for Enterprise Linux. Enabled by default.
Packages published in above repositories should always allow a non-disruptive automatic update.
YUM repositories configuration¶
Two special .conf
files control how NethServer configures and invokes YUM:
/etc/nethserver/pkginfo.conf
: list of YUM repositories that have their groups listed- on the Software Center
/etc/nethserver/eorepo.conf
: list of YUM repositories enabled bysoftware-repos-save
event, every non-listed repository will be disabled
This is the list of .repo
files providing the default repositories
configuration:
/etc/yum.repos.d/NethServer.repo
ce-base
ce-updates
ce-extras
ce-sclo-sclo
ce-sclo-rh
nethserver-base
nethserver-updates
/etc/yum.repos.d/epel.repo
epel
/etc/yum.repos.d/NethForge.repo
nethforge
ce-*
, nethserver-*
and nethforge
repositories are accessed with a
full release number (e.g. 7.6.1810), preventing unwanted upgrades to the next
minor release version. See the software-repos-upgrade
event for details.
The EPEL repository does not support accessing RPMs using a minor release like
7.5.1804
but only using a major release like 7
.
Warning
When a subscription is enabled the default repositories are disabled. See
/etc/yum.repos.d/subscription.repo
Third party repositories¶
It’s possible to install third party repositories, using standard CentOS methods.
If such repositories support access using minor release, they can be safely added
to eorepo.conf
and pkginfo.conf
using a template-custom.
yum-cron¶
Since NethServer 7.5.1804
, nethserver-yum-cron
has been merged into nethserver-base
.
The cron job runs each night with a random time before to start of 6 hours..
You can decide who receive the notifications (default is root), which updates to do, if you just check, download, or install automatically the updates.
Original author: Stephane de Labrusse (@stephdl)
Database¶
Properties:
applyUpdate
: can beyes
orno
. If set toyes
, downloaded updates will be installedcustomMail
: comma-separated list of extra mail recipients, as default a mail will be sent to rootdownload
: can beyes
orno
. If set toyes
, download new package updatesmessages
: can beyes
orno
. Whether a message should be emitted when updates are availablerandomWait
: random number of minutes to wait before executing the download procedure - NS6: 1 to 60 minutes - NS7: 1 to 360 minutes, negative and the job start immediatelystatus
: can beenabled
ordisabled
. When enabled, a cron script will search for package updates
Database example:
yum-cron=service
applyUpdate=yes
customMail=
download=no
messages=no
randomWait=360
status=enabled
Notifications¶
Mail for root user can be forwarded to external addresses.
Properties:
- EmailAddress: comma-separated list of mail addresses; messages sent to root user will be forwarded to listed addresses
- KeepMessageCopy: can be yes or no; if set to yes, messages will be always delivered also to local root mail folder
- SenderAddress: a valid mail address; if not empty, messages sent by root (like cron notifications) will be sent using the specified address. A good value could be: no-reply@<domain> (where <domain> is the domain of the server). If not set, messages will be sent using root@<fqdn> as sender address.
Database example:
root=configuration
EmailAddress=myuser@nethserver.org
KeepMessageCopy=yes
SenderAddress=no-reply@nethserver.org
Usage:
config setprop root EmailAddress myuser@nethserver.org SenderAddress no-reply@nethserver.org
signal-event notifications-save
Let’s Encrypt certificates¶
This package also requests and automatically renews Let’s Encrypt (LE) certificates. It adds httpd ACME-related configuration for all defined virtual hosts.
The main helper /usr/libexec/nethserver/letsencrypt-certs
can be executed also from command line.
For more info, see:
/usr/libexec/nethserver/letsencrypt-certs -h
Database properties under pki
key inside configuration
database:
LetsEncryptRenewDays
: days to the expiration, the certificate will be renewd when the condition is metLetsEncryptMail
: (optional) registration mail for LE notificationsLetsEncryptDomains
: comma-separated list of domains added to certificate SAN fieldLetsEncryptChallenge
: challenge to use for validating the certificate, default ishttp
. It accepts also values likedns-<provider>
. Where<provider>
is the name of the DNS provider. See the full list of available DNS provider plugins by executingcertbot -h certonly
. More info at https://certbot.eff.org/docs/using.html?highlight=dns#dns-plugins.LetsEncryptShortChain
: can beenabled
ordisabled
. Ifenabled
, the certificate chain will not contain the expired DST Root CA X3
DNS challenge¶
To use the DNS challenge, follow these steps:
- install the required certbot plugin plugin using yum; to see the list of available package use
yum search certbot-dns
- set
LetsEncryptChallenge
property to correct DNS plugin - configure all required properties accordingly to plugin documentation
When using the dns challenge, make sure to set extra properties accordingly to certbot configuration.
All properties for the dns challenge should be in the form LetsEncrypt_<certbot_option>
, where
<certbot_option>
is the option specific to certbot DNS plugin.
Digitial Ocean example¶
Install the plugin:
yum install python2-certbot-dns-digitalocean
Configure the challenge type:
config setprop pki LetsEncryptChallenge dns-digitalocean
Configure required props accordingly to https://certbot-dns-digitalocean.readthedocs.io/en/stable/:
config setprop pki LetsEncrypt_dns_digitalocean_token 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
Request certificate for domain
myserver.nethserver.org
:/usr/libexec/nethserver/letsencrypt-certs -v -d myserver.nethserver.org
nethserver-blacklist¶
nethserver-blacklist
provides two kinds of blacklists: IP blacklist and DNS blacklist.
The former blocks connections from/towards a set if IP addresses, while the latter blocks DNS requests to a set of domains.
IP blacklist¶
Configuration¶
IP blacklist is managed by properties of blacklist
key inside configuration
database:
status
: can beenabled
ordisabled
, ifenabled
selectedCategories
will be blocked using ipsetsCategories
: a comma-separeted list of blacklist categories, a valid category must have a corresponding file inside/usr/share/nethserver-blacklist/ipsets
. Example: given the categorytest
, a file named/usr/share/nethserver-blacklist/ipsets/test.netset
or/usr/share/nethserver-blacklist/ipsets/test.ipset
must existUrl
: the GIT URL from where blacklists will be downloadedWhitelist
: a comma-separated list of hosts excluded from the blacklists. The host can be an IP, a CIDR, an host object or a CIDR objectMaxElem
: It does define the maximal number of elements which can be stored in the set, default 131072. You have to stop shorewall before to destroy the set, the shorewall service start will created it again with the new value.
config setprop blacklist MaxElem 262144
systemctl stop shorewall
for S in $(ipset -L -name | grep '^bl-'); do : ; ipset destroy $S; done
systemctl start shorewall
Example:
blacklist=configuration
Categories=category1
Url=https://my.nethserver.org/git/myrepo
Whitelist=
status=disabled
Blacklists¶
Blacklist implementation is based on ipset:
- download ipset-based blacklist from a remote GIT repository
- block access from/to black listed IPs
- support whitelisting
- lists are updated every 20 minute, ipsets are reloaded on change
- all blocked IPs are logged inside
/var/log/firewall.log
GIT repository¶
The remote GIT repository may contain one or more files with .netset
or .ipset
using Firehol iplists format,
each file represent a category.
In addition to Firhol file format, the files can contain a Confidence
tag inside the header. Example:
# ...
# Category : attacks
# Version : 18
# Confidence : 8
# ...
The confidence is a number between 0
and 10
, a higher number means less false positives.
Lists with a confidence greater than 8, should be safe for production.
The repository can also contain a special file named whitelist.global
which is a special netset that
can be used to quickly remove false positive from a blacklist.
Setup a blacklist server¶
You can build your own server to distribute blacklists based on Firehol iplists. First install a clean NethServer, then follow the below steps.
Install required packages:
yum install https://github.com/firehol/packages/releases/download/2021-01-01-1948/firehol-3.1.7-11.el7.noarch.rpm https://github.com/firehol/packages/releases/download/2021-01-01-1948/iprange-1.0.4-2.el7.x86_64.rpm
yum install https://repo.ius.io/ius-release-el7.rpm
yum install -y git224-core --enablerepo=ius
Create the git repository and serve it using Apache:
cat << EOF >> /etc/httpd/conf.d/git.conf
SetEnv GIT_PROJECT_ROOT /var/www/html/git/
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
<Directory "/usr/libexec/git-core/">
<Files "git-http-backend">
Options +ExecCGI
Require all granted
</Files>
</Directory>
EOF
systemctl restart httpd
Configure and enable update-ipsets
:
cat << EOF > /etc/firehol/update-ipsets.conf
BASE_DIR=/var/www/html/git/ipsets
HISTORY_DIR=/var/www/html/git/ipsets/history
ERRORS_DIR=/var/www/html/git/ipsets/errors
GIT_CMD=/usr/bin/git
EOF
mkdir -p /var/www/html/git/ipsets
git -C /var/www/html/git/ipsets init
/usr/sbin/update-ipsets enable dshield feodo fullbogons spamhaus_drop spamhaus_edrop sslbl zeus_badips ransomware_rw firehol_level1
/usr/sbin/update-ipsets run dshield feodo fullbogons spamhaus_drop spamhaus_edrop sslbl zeus_badips ransomware_rw firehol_level1
Set up a cron to regularly update the ipsets:
cat << EOF >> /etc/cron.d/update-ipsets
*/19 * * * * root /usr/sbin/update-ipsets
EOF
See also https://github.com/firehol/blocklist-ipsets/wiki/downloading-ip-lists
DNS blacklist¶
DNS blacklist uses Pi-Hole FTLDNS under the hood.
Configuration¶
DNS blacklist is managed by properties of ftl
key inside configuration
database:
status
: can beenabled
ordisabled
, ifenabled
selectedCategories
will be blocked using FTLDNSCategories
: a comma-separeted list of blacklist categories. Valid categories have a corresponding file inside/usr/share/nethserver-blacklist/dnss
Example: given the categorytest
, a file named/usr/share/nethserver-blacklist/dnss/test.dns
must existUrl
: the GIT URL from where blacklists will be downloadedBypass
: a comma-separated list of hosts whose DNS requests are always allowed. A host can be an IP, a CIDR, an host object or a CIDR objectRoles
: a comma-separated list of firewall zones where DNS blacklist is enabledUDPPorts
,TCPPorts
: the ports FTLDNS is listening toaccess
: the zonesftl
systemd service has access to
Example:
ftl=service
Bypass=
Categories=category1,category2
Roles=green
TCPPorts=1153
UDPPorts=1153
Url=https://my.nethserver.org/git/myrepo
access=green
status=disabled
Blacklists¶
Blacklist implementation is based on Pi-Hole gravity database:
- download DNS blacklists from a remote GIT repository and insert them into gravity database
- block DNS requests for listed domains
- support bypass
- lists are updated every 20 minute, ipsets are reloaded on change
- configuration process is logged inside
/var/log/pihole-FTL.log
- blocked requests and other statistics are available through FTLDNS telnet API
GIT repository¶
The remote GIT repository may contain one or more files with .dns
extension listing domains to block. Each file represents a category. Category files can contain Maintainer
, Category
and Confidence
tag inside the header. Confidence
is a number between 0
and 10
, a higher number means less false positives.
Lists with a confidence greater than 8
should be safe for production.
Example content:
#
# Maintainer : John Doe
# Category : Malware
# Confidence : 6
#
unwanted.domain.com
dangerousdomain.net
malwaresite.net
...
GEOIP blacklist¶
Configuration¶
GEOIP blacklist is managed by properties of geoip
key inside configuration
database:
status
: can beenabled
ordisabled
, ifenabled
selectedCategories
will be blocked using ipsetsCategories
: a comma-separeted list of geoip blacklist categories, a valid category must have a corresponding file inside/usr/share/nethserver-blacklist/geoips
without the extensionsnetset
. Example: given the categoryfr.netset
, a file named ``/usr/share/nethserver-blacklist/geoips/fr.netset``must existUrl
: the GIT URL from where blacklists will be downloaded (default ipdeny.com)Whitelist
: a comma-separated list of hosts excluded from the blacklists. The host can be an IP, a CIDR, an host object or a CIDR objectMaxElem
: It does define the maximal number of elements which can be stored in the set, default 131072. You have to stop shorewall before to destroy the set, the shorewall service start will created it again with the new value.
config setprop geoip MaxElem 262144
systemctl stop shorewall
for S in $(ipset -L -name | grep '^geo-'); do : ; ipset destroy $S; done
systemctl start shorewall
Example:
geoip=configuration
Categories=fr,es
Url=https://www.ipdeny.com/ipblocks/data/countries/all-zones.tar.gz
Whitelist=
status=disabled
blacklist¶
Geoip blacklist implementation is based on ipset:
- download geoip blacklist from the website ipdeny.com
- block access from/to black listed IPs
- support whitelisting
- lists are updated each night, ipsets are reloaded on change
- all blocked IPs are logged inside
/var/log/firewall.log
Example¶
The geoip list must be enabled manually:
config setprop geoip status enabled
signal-event nethserver-blacklist-save geoips
Once downloaded you can enable the geo banning by the command line:
config setprop geoip status enabled Categories es,fr Whitelist '195.234.41.0/24,195.234.42.3'
signal-event nethserver-blacklist-save geoips
The command above takes care to whitelist a CIDR network or a specific IP
nethserver-cgp¶
Expose collectd metrics using Collectd Graph Panel (CGP): https://github.com/pommi/CGP
Database example:
cgp=configuration
alias=02bf3f8364beea0d5f23044bf14d31d93f63e98d
The alias
is a random path generated once on first install and it will be used to compose CGP URL.
So in this case will be acessibile from https://<fqdn>:980/02bf3f8364beea0d5f23044bf14d31d93f63e98d
.
Where <fqdn>
is the fully qualified domain name of the server or its IP address.
nethserver-cockpit¶
This module implement new Server Manager UI using Cockpit project.
API and NethServer Cockpit developer guide: https://nethserver.github.io/nethserver-cockpit/
nethserver-collectd¶
This package automatically configure basic collectd plugins and it’s part of nethserver-statistics
yum package group.
This package will install a base collectd configuration. Default enabled modules:
- cpu
- load
- processes
- memory
- swap
- uptime
- df
- disk
- interface
- network
- rrdtool
- ping
No configuration is needed, collectd is enabled by default when installed.
Ping plugin¶
The ping plugin sends an ICMP packet every 5 seconds to: * upstream DNS * every checkip of enabled provider (see Multi WAN)
Additional hosts must be added to the PingHosts
property:
config setprop collectd PingHosts www.nethesis.it,www.nethserver.org
signal-event nethserver-collectd-update
Web interface¶
To view graphs of collected data, install and nethserver-cgp.
Cleanup¶
Every day a cronjob (/etc/cron.daily/collectd_cleanup
) takes care to clean up all RRD files not updated
during the last day.
Database¶
Configuration is saved inside the configuration
database. Example:
collectd=service
PingHosts=
status=enabled
nethserver-cups¶
CUPS is the printer server. It converts the page descriptions produced by your application (put a paragraph here, draw a line there, and so forth) into something your printer can understand and then sends the information to the printer for printing.
Configuration¶
Use the web configuration tool: http://<server>:631
You’ll need to create 2 printers:
- Raw printer for use with Samba and Windows hosts
- A PPD defined printer to user via IPP and Linux hosts
You already have a PDF printer configured and ready to use. All files printed via PDF Printer are saved in the user home directory, inside the directory called “Desktop”. You can download the files using samba.
Printer discovery¶
CUPS uses avahi to discover network-attached printer. If you wish to use this features, just install the optional package nethserver-avahi. See avahi-section.
Configuration DB¶
Reference:
cups=service
TCPPort=631
UDPPort=631
access=private
status=enabled
Links
- Offical CUPS web site: http://www.cups.org/
- Offical AVAHI web site: http://avahi.org/
nethserver-dc¶
The nethserver-dc package runs a systemd-nspawn container (nsdc
) with a vanilla
Samba 4 inside of it. It downloads, installs, configures and provision an Active
Directory domain controller based on Samba.
The nsdc
container needs an IP address in a green network, different from the
host machine one. It enslaves its network interface to a host bridge, with green
role. If needed, this bridge is created automatically.
This is a typical configuration:
# config show nsdc
nsdc=service
ProvisionType=newdomain
IpAddress=192.168.122.50
bridge=br0
status=enabled
nethserver-dc-save event¶
- it creates and configures systemd-nspawn machine (nethserver-dc-install
action). The Samba domain is provisioned by the
samba-provision.service
unit, according to theProvisionType
prop value. Supported values are:newdomain
(default): domain and realm are taken from local system and won’t be possible to change them anymore. For instance if system domain is nethserver.org domain will be NETHSERVER and realm nethserver.org.ns6upgrade
: connect the LDAP service running on the host machine and migrate the WS/PDC domain from ns6 backup to an Active Directory domain. The realm and domain name are set as described in thenewdomain
provision type.sme8migration
: this provision procedure is driven bynethserver-dc-migrate
action. It requires a running NSDC, provisioned asnewdomain
. It copies Samba 3 files from the migration source directory to the NSDC chroot, then restarts thesamba-provision
andsamba
services in the running NSDC container.
- it creates a network bridge if needed, or select an existing one and save it in nsdc bridge db prop (nethserver-dc-create-bridge action)
- it waits for the machine to come up (nethserver-dc-waitstart)
- it joins the domain of new machine using default credentials (nethserver-dc-join).
- it sets the password policy (nethserver-dc-password-policy)
Realmd writes a lot of information on the system journal. See journalctl command.
Running commands in nsdc
¶
To get a shell inside the nsdc
container, you can run
# systemd-run -M nsdc -t /bin/bash
As alternative, run a specific command in the container with the nsdc-run
helper:
# nsdc-run cat /etc/hostname
The --
string helps to separate the argument list:
# nsdc-run -- ls --help
For more information:
# nsdc-run --help
Always make sure to close interactive sessions to avoid errors like:
Failed to get machine PTY: No such file or directory
To get a list of sessions run
loginctl
Manual Join¶
nethserver-dc-join action joins automatically to domain. If for any reason the join is invalid you can attempt a manual join following this procedure
Check nsdc is running:
systemctl status nsdc
Check the DNS is responding:
# host -t SRV _ldap._tcp.$(config getprop sssd Realm)
_ldap._tcp.nethsever.org has SRV record 0 100 389 nsdc-vm8.nethsever.org.
Clean up any previous join state:
config setprop sssd Provider none
signal-event nethserver-sssd-leave
Join the domain:
realm join -v -U admin $(config getprop sssd Realm)
You can replace admin
with any other administrative account name. The
command above prompts for a password. When join is successful:
config setprop sssd Provider ad
signal-event nethserver-sssd-save
If everything goes well
getent passwd administrator@$(hostname -d)
# output: administrator@nethserver.org:*:261600500:261600513:Administrator:/home/administrator@nethserver.org:/bin/bash
/usr/libexec/nethserver/list-users -s administrator
# output: {"administrator": ...
Once domain is joined, you can manage users from interface. From command line, you can use net command
# net ads info
Factory reset¶
The “Start DC” procedure from the “Accounts provider” page is designed for a single run. If it fails, re-installing the whole server can be avoided by running the following command
signal-event nethserver-dc-factory-reset
The command cleans up the DC state and prepare it for new provisioning run. Any existing user and group account is erased.
If a full DC re-install is desired, after factory reset event, run also
rm -rf /var/lib/machines/nsdc
Upgrade the container¶
The upgrade procedure will:
- stop the container
- upgrade the chroot base system
- upgrade samba
- restart the container
To upgrade, execute:
signal-event nethserver-dc-upgrade
Changing the IP address of DC¶
Warning
Before applying this procedure, read carefully the official Samba wiki page.
The IP address of nsdc container must be in the same network of the bridged green interface. If needed, first change the address of the green interface, then proceed with the following.
Example, change the network address:
- current host IP: 192.168.101.7
- current nsdc container IP: 192.168.122.77
- new nsdc container IP: 192.168.101.77
Execute the nethserver-dc-change-ip
with the new ip address:
signal-event nethserver-dc-change-ip <new_ip_address>
Example:
signal-event nethserver-dc-change-ip 192.168.101.77
Note that the event will fail if the new nsdc ip address is not in the same network of the green interface.
Alternate UPN suffix¶
The default UPN (User Principal Name) suffix for a user account is the SSSD realm, but the nsdc container is configured to use also an extra UPN suffix set to the FQDN of the host machine.
Example:
- Host FQDN: nethserver.org
- SSSD realm: ad.nethserver.org
- Default UPN: ad.nethserver.org
- Extra UPN: nethserver.org
If required, the administrator can use RSAT tools to select the extra UPN for a specific user.
References:
Enable check password script¶
This manual fix is only needed for old installations.
Edit /var/lib/machines/nsdc/etc/samba/smb.conf
and add
check password script = /usr/local/sbin/checkpassword.pl
to the global section. Restart samba on the NSDC container:
systemctl -M nsdc restart samba
Apply the current system password policy to the Samba DC:
signal-event password-policy-update
ns-samba binary build¶
Clone the git repo at https://github.com/NethServer/ns-samba, enter its directory
Download pristine sources from the Samba project
wget https://download.samba.org/pub/samba/stable/samba-4.8.6.tar.gz https://download.samba.org/pub/samba/stable/samba-4.8.6.tar.asc
Check the sources signature
gunzip -c samba-4.8.6.tar.gz | gpg --verify samba-4.8.6.tar.asc -
Bump the source hash sum
sha1sum samba-4.8.6.tar.gz > SHA1SUM
Edit
ns-samba.spec
to fix version references and%changelog
section
- Commit changes and add (signed) tag (i.e.
4.8.6
) - Push commit and tag to remote git repository to start the automated x86_64 build
- Once the automated build has finished successfully, fill the new relase page here: https://github.com/NethServer/ns-samba/releases
- Copy the RPM URL to the local
nethserver-dc.spec
file - Update SHA1SUM with the new RPM hash sum
nethserver-directory¶
The nethserver-directory implements user authentication and authorization. All accounts are saved inside OpenLDAP.
Features:
- RFC2307 schema, user and group account management
- PAM LDAP password storage
- Password strength control
- Service accounts
Schema and base DN¶
Following schema are always loaded by OpenLDAP: core, cosine, nis, inetorgperson.
The LDAP tree is always accessible with the following base DN: dc=directory,dc=nh. But there is also an overlay which maps the domain name to the base DN. For example, given the domain mydomain.com, the corresponding DN will be dc=mydomain,dc=com.
Accounts are saved inside following branches:
- Users: ou=People,dc=directory,dc=nh
- Groups: ou=Groups,dc=directory,dc=nh
All users are in the primary group named locals.
Examples¶
List all entries, with root access and automatic bind (unix domain socket):
ldapsearch -Y EXTERNAL
List all entries with libuser bind:
ldapsearch -D cn=libuser,dc=directory,dc=nh -w `cat /var/lib/nethserver/secrets/libuser` -h 127.0.0.1
User account states¶
Account management is done via libuser which exports some tools (luseradd, luserdel, etc) to create/modify/delete users, groups and set passwords.
The process of user/group creations is:
- invoke the create event which uses libuser to add the record inside LDAP
- invoke the group event if the user must be member of additional groups
- invoke the password policy event to change user password expiration
- invoke the password change event to set a password
Logging¶
OpenLDAP doesn’t output any log with standard configuration.
When logging is enabled, all logs are saved inside /var/log/slapd
.
But its verbosity can be changed at run time by issuing this command:
# ldapmodify -Y EXTERNAL <<EOF
dn: cn=config
changetype: modify
replace: olcLogLevel
olcLogLevel: 256
EOF
The command above changes the OpenLDAP config DB and sets the log verbosity to trace “connections/operations/results” (256). Check the debugging levels table from OpenLDAP site for more details: http://www.openldap.org/doc/admin24/slapdconf2.html#olcLogLevel%3A%20%3Clevel%3E.
Service accounts¶
The following service accounts are configured by the nethserver-directory-dit-setup
action:
cn=libuser,dc=directory,dc=nh
. Granted read and write access from 127.0.0.1.cn=ldapservice,dc=directory,dc=nh
. Granted read only access to non-sensitive attributes, from localhost, or from any other IP address with TLS.
The developer can use the NethServe::Directory
Perl module to handle
additional service accounts with ad-hoc permissions, if the existing ldapservice
,
libuser
accounts and anonymous binds do not fit his requirements.
A service account is composed by three parts:
- a LDAP user
- a password
- an ACL to access LDAP fields
Perl code snippet to create a service account with read access:
use NethServer::Directory;
...
NethServer::Directory->new()->configServiceAccount('myservice', NethServer::Directory::FIELDS_READ) || die("Failed to register myservice account")
Perl code snippet to use created password:
use NethServer::Password;
my $pwd = NethServer::Password::store('myservice');
User accounts¶
Authenticated binds are granted to TLS protected connections, or connections from 127.0.0.1. User DN are in the form:
uid=<username>,ou=People,dc=directory,dc=nh
Anonymous access¶
Some LDAP clients and/or legacy environments may require anonymous bind to the LDAP accounts database. Currently anonymous access is granted to non-sensitive fields.
Configuration for client:
- Host: ip address of the server
- Port: 389
- Base DN:
dc=directory,dc=nh
Administrative access¶
An existing DN (i.e. administrator
) can be granted full administrative
privileges on the whole dc=directory,dc=nh
tree. By default, the designated
user is defined in config DB, under the admins
key.
`
ldapmodify -Y EXTERNAL <<EOF
dn: olcDatabase={2}hdb,cn=config
changetype: modify
replace: olcRootDN
olcRootDN: uid=administrator,ou=People,dc=directory,dc=nh
EOF
`
Inspect OpenLDAP ACLs¶
Service accounts require OpenLDAP ACLs tuning. To inspect the current ACLs type:
ldapsearch -LLL -Y EXTERNAL -b cn=config -s one 'objectClass=olcDatabaseConfig' olcAccess 2>/dev/null
If output appears to be base64-encoded type:
ldapsearch -LLL -Y EXTERNAL -b cn=config -s one 'objectClass=olcDatabaseConfig' olcAccess 2>/dev/null | perl -MMIME::Base64 -MEncode=decode -n -00 -e 's/\n +//g;s/(?<=:: )(\S+)/decode("UTF-8",decode_base64($1))/eg;print'
Upgrade to Active Directory¶
If the LDAP database has been restored from a ns6 backup set, it is possible to upgrade it to a local Active Directory accounts provider.
A ns7 LDAP database cannot be upgraded to Active Directory. It lacks the Samba LDAP schema extensions required by the Samba classic upgrade procedure.
The nethserver-directory-ns6upgrade
event
- removes the
nethserver-directory
RPM - installs and configures
nethserver-dc
Before running the event, assign a free IP address to the nsdc
Linux
container, installed by nethserver-dc
RPM. Ensure it is a free IP
address of a green network.
config set nsdc service IpAddress A.B.C.D
signal-event nethserver-directory-ns6upgrade
nethserver-dnsmasq¶
DNS¶
The system will resolve host and domain names using DNS queries
to external DNS servers.
The configuration is saved inside the dns
key from nethserver-base package.
Properties:
NameServers
: comma separated IP list of external DNSrole
: can be set tonone
orresolver
. If role is set tonone
the server will always use external DNS. Forresolver
role see DNS server.
Example:
dns=configuration
NameServers=8.8.8.8,208.67.222.222
role=none
Hosts¶
The system can handle local DNS records. When the server performs a DNS lookup, first it will search inside local DNS records. If no local record is found, an external DNS query will be done.
Note
Local DNS record will always override records from external DNS servers.
DNS records are called hosts and are saved inside the hosts
database.
Each entry is saved inside the /etc/hosts
file.
There are three types of records:
local
: hosts inside the internal networkremote
: hosts outside the internal networkself
: alias for the server itself
Records of type local
and remote
can have following properties:
IpAddress
: address of the hostDescription
: optional descriptionMacAddress
: mac address of the host. Used only for DHCP reservation. See IP reservation.
For hosts inside local network, the record key doesn’t have the domain part. Example:
host1=local
Description=Internal network host #1
IpAddress=192.168.1.23
For hosts outside local network, the record key must have the domain part. Example:
external.otherdomain.tld=remote
Description=Other domain host
IpAddress=8.9.10.11
Records of type self
can have following properties:
Description
: optional description
Example:
vhost1.domain.tld=self
Description=Virtual Host #1
DNS server¶
The system uses dnsmasq as DNS and DHCP server and it directly resolves all hosts inside its domain. All other names will be queried to external DNS servers.
The server will forward reverse lookups to upstream DNS servers, only if upstream DNS servers are inside a private network (eg. network address is 192.168.x.x).
The option bind-interfaces
is always enabled, as consequence (from dnsmasq man):
This option has been patched to always use SO_BINDTODEVICE socket option when binding to interfaces. As consequence, dnsmasq WILL NOT ANSWER to any DNS Queries that come to the socket with the correct destination IP address, but originally on different interface. This behavior differs from the original dnsmasq upstream version and is used for security reasons.
Properties:
CacheSize
: entry to be cached by server, default is4000
dhcp-boot
: directly pass parameters to dhcp-boot optionexcept-interface
: comma-separated list of interfaces. Do not listen to listed interfaces, useful to avoid conflicts with libvirttftp-status
: can beenabled
ordisabled
. If enabled, enable the TFTP server for BOOTP (port 67)access
: default isprivate
, do NOT set topublic
DomainRedirection
: specify a dns server for a particular domain (comma separated). Thedomain.org:192.168.1.1
will send all queries*.domain.org
for internal machines to192.168.1.1
. The special server address#
means, “use the standard servers”, sosub.domain.org:#
will send all queries for*sub.domain.org
to the default DNS server of the domain name.
Database example:
dnsmasq=service
AllowHosts=
CacheSize=4000
DenyHosts=
DomainRedirection=domain.org:192.168.1.1,sub.domain.org:#
TCPPort=53
UDPPorts=53,67
access=private
dhcp-boot=pxelinux.0,myserver.mydomain.com,192.168.1.1
except-interface=virbr0,tunspot
status=enabled
tftp-status
DHCP¶
The system can act as DHCP server for the local network. Machines which are configured by DHCP have their names automatically included in the DNS server.
The DHCP can be enabled only on green and blue interfaces (see Roles and zones).
Configuration is saved inside the dhcp
database.
Each record of range
type is associated to an ethernet interface and can have following properties:
status
: can beenabled
ordisabled
DhcpRangeStart
: first IP address of DHCP rangeDhcpRangeEnd
: last IP address of DHCP rangeDhcpLeaseTime
: seconds of lease time. Default is 86400DhcpGatewayIp
: (optional) set a custom gateway ip. If not set, the gateway is the ip address of associated interface (record key)
The key of the record is the name of the associated interface. Example:
eth0=range
DhcpGatewayIp=
DhcpLeaseTime=86400
DhcpRangeEnd=192.168.1.100
DhcpRangeStart=192.168.5.200
status=enabled
Hosts inside the blue network can always access the local DNS server.
The gateway for clients will be:
- if set, the value of property
DhcpGatewayIp
- otherwise if the server has a red interface, the gateway is the IP address of the interface where the DHCP is enabled (eg. IP of the blue interface for clients in the guest’s network)
- otherwise if the server has only a green interface, the gateway of the green interface will be used
IP reservation¶
It’s possible to reserve IPs for specific devices associating the MAC address of the device with the reserved IP.
The reservation is saved inside the hosts
database.
Example:
host1=local
Description=Internal network host #1
IpAddress=192.168.1.23
MacAddress=08:00:27:48:BF:F3
TFTP server¶
TFTP module contains configuration fragments that enables dnsmasq built-in TFTP server.
TFTP server has no authentication or encryption support.
When installed tftp is disabled by default and need to be enabled with:
config setprop dnsmasq tftp-status enabled
signal-event nethserver-dnsmasq-save
The package also add directory /var/lib/tftpboot
that is the root of tftp server.
Enabling TFTP adds 5 new configuration options to /etc/dnsmasq.conf
. Here variables explanation according with dnsmasq documentation
enable-tftp
: enable tftp servertftp-secure
: allow only files owned by the user dnsmasq is running as will be send over the netdhcp-boot= ...
: Set the boot filename for netboot/PXE. You will only need this is you want to boot machines over the network and you will need a TFTP server; driven by db proptftp-root=/var/lib/tftpboot
: Set the root directory for files available via FTP.dhcp-option=66, LOCAL_IP
: set local ip as default tftp server for machines that receive dhcp from this server
Properties¶
status
: can beenabled
ordisabled
. Ifenabled
, TFTP server is configured and port 69 UDP is opened.UDPPort
: UDP port used. Only69
is allowed.access
: define if access ispublic
,private
ornone
.dhcp-boot
: Set the boot filename for PXE. Ths is needed for booting machines over the network. Empty by default.type
: onlyservice
is allowed.
Test TFTP¶
Testing is very simple:
Enable TFTP server:
config setprop dnsmasq tftp-status enabled
signal-event nethserver-dnsmasq-save
Create a file to share, owned by nobody
user:
echo "test" > /var/lib/tftpboot/foobar
chown nobody:nobody /var/lib/tftpboot/foobar
From another machine, install tftp and get file (on Fedora):
yum install tftp
Always from the other machine, allow incoming UDP connection from our TFTP server. Loading TFTP conntrack module should be enough:
modprobe nf_conntrack_tftp
Connect to TFTP server:
tftp TFTP_SERVER_HOST
…and get the file:
tftp> get foobar
Configure a PXE server¶
Those instructions set up a PXE server for CentOS Install and configure syslinux and nethserver-tftp:
yum install syslinux
cp /usr/share/syslinux/{pxelinux.0,menu.c32,memdisk,mboot.c32,chain.c32} /var/lib/tftpboot/
config setprop tftp dhcp-boot pxelinux.0
signal-event nethserver-tftp-save
mkdir /var/lib/tftpboot/pxelinux.cfg
Create the file /var/lib/tftpboot/pxelinux.cfg/default
with the following content:
default menu.c32
prompt 0
timeout 300
MENU TITLE PXE Menu
LABEL CentOS
kernel CentOS/vmlinuz
append initrd=CentOS/initrd.img
Create a CentOS directory:
Create a CentOS directory:
mkdir -p /var/lib/tftpboot/CentOS
Copy inside the directory vmlinuz
and initrd.img
files. These files can be found inside the ISO or browsing the yum os
mirror.
Change files owner to nobody:
chown -R nobody /var/lib/tftpboot/*
nethserver-duc¶
Official site: https://github.com/zevv/duc
This tool is used to visualize disk usage in a simply and nice graph.
Features:
- Indexing filesystem very fast
- Interact with graph with click and double click to navigate in the directories tree.
Overview¶
This tool is composed by two parts: duc indexer and duc viewer. The first one is launched every day and it indexes the /
directory and it creates a file that is loaded by duc viewer to show graph.
In the graph are showed only directories over certain size, in particularly this size is 10MB as you can see in file /etc/e-smith/events/actions/nethserver-duc-index
.
Configuration¶
By default, duc will skip directories on different file systems. To change this behavior and include all filesystems execute:
config setprop duc OneFileSystem disabled
nethserver-ejabberd¶
The chat function is implemented using ejabberd XMPP server. Enabled features are:
- LDAP based roster
- SSL/TLS
- Admin group
If you want to give admin permissions to an existing user, just add the user to the special group jabberadmins
.
The jabberadmins
must be created manually.
If the system has a remote account provider also reconfigure the ejabberd service:
sss_cache -g jabberadmins
signal-event nethserver-ejabberd-save
When used with AD backend, following limitations apply:
- The shared roster doesn’t support groups
- The shared roster displays the list of user names (not full names)
See also the Server Manager UI API documentation: https://github.com/NethServer/nethserver-ejabberd/blob/master/UI-API.md
Configuration¶
Properties:
- WebAdmin: Enable ejabberd built-in web interface. Can be
enabled
ordisabled
, default isdisabled
- S2S: Enable the server-to-server (S2S) for XMPP federation (Port number: 5269). Can be
enabled
, default isdisabled
- ModMamStatus: The XEP-0313: Message Archive Management (mod_mam). Can be
enabled
, default isdisabled
- ModMamPurgeDBStatus: Purge the Mnesia database of old messages of mod_mam
- ModMamPurgeDBInterval: Remove messages older than X days, default is
30
- ShaperFast: Download speed limit in bytes/second for admin users, default is
1000000
- ShaperNormal: Download speed limit in bytes/second for users, default is
500000
- LdapTlsPortOverride: The user roster LDAP client does not support STARTTLS. If the system account provider is configured with STARTTLS, configure the user roster LDAP client to use this TCP port number during connections instead (default is port 636).
When enabled, web-based administration interface listens on 5280 port. You need a user inside jabberadmins group to login.
Default access to server ports is set to public on following ports: 5280, 5222, 5223.
The XMPP server can be accessed using BOSH protocol (https://xmpp.org/extensions/xep-0206.html) at URL /http-bind
.
Example:
- server FQDN: mail.nethserver.org
- BOSH URL: https://mail.nethserver.org/http-bind
nethserver-evebox¶
EveBox configuration for NethServer:
- it requires nethserver-suricata
- it runs as
suricata
user, only if/var/log/suricata/eve.json
file exists - Suricata must be running and eve log must be enabled
- it uses SQLite
- no ElasticSearch support
- listens on port 5636, accessible only from localhost
- web interface served via reverse proxy:
https://<host>:980/<alias>
(seealias
prop below) - once a week internal GeoIP database is updated by a cron job
Properties:
- Retention: positive integer, if greater than 0, events will be discarded after the given number of days
Database:
evebox=service
TCPPort=5636
access=
alias=be771675da40887ccf9eb98d05b88dc80559b6a6
Retention=30
status=enabled
Links¶
- Official site: https://evebox.org/
nethserver-firewall-base¶
NethServer can work into two basic modes:
- server mode: the system will be a standard host inside the network offering services like e-mail or file server.
- gateway mode: the system is the gateway and firewall of the local network
The system has an abstraction layer for firewall base functions, like opening ports for running services. Actually two implementations are available:
- server mode: standard lokkit (default on CentOS)
- gateway mode: advanced Shorewall configuration
The gateway functionality is built around three modules:
- nethserver-base: high-level abstraction of the firewall
- nethserver-firewall-base: Shorewall-based implementation
- nethserver-lsm: link status monitor for multi-wan configurations
Roles and zones¶
Each network interface has a role which maps to a firewall zone. The firewall has the following built-in zones, ordered from the most to the least privileged:
- green: local network, it’s considered almost trusted. Hosts in this network can access any other zone. Hosts connected via VPN can be considered in green zone.
- blue: guest network. Hosts in this network can access orange and red zones, can’t access green zone
- orange: DMZ network. Hosts in this network can access red zone, can’t access green and blue zones
- red: external/internet networks. Hosts in this network can access only firewall zone
There is also a special firewall zone which represents the firewall itself. The firewall can access any other zone.
Each network interface with a configured role is a firewall zone. Roles are mapped to Shorewall zones as:
- green -> loc
- red -> net
- blue -> blue
- orange -> orang (in Shorewall, a zone name can’t be longer than 5 chars)
- firewall -> FW
Custom zone names are directly mapped to Shorewall respecting the limit of 5 chars.
Red interfaces can be configured with static IP address or using DHCP. All other interfaces can be configured only with static IP addresses.
General configuration¶
Properties of firewall
key inside configuration
db:
event
: event to call whenfirewall-adjust
event is firedExternalPing
: if enabled, allow ping responses on external interfaceWanMode
: multi-wan mode. Default isbalance
, can be:balance
: traffic is balanced among red interfaces in weighted modebackup
: traffic is routed via wan interface with maximum weight, all other interfaces are used as fallback
nfq
: if enabled, traffic from external networks will be passed to NFQ and scanned with Snort. See ips.Policy
: can bepermissive
orstrict
. See above.MACValidation
: if enabled, the firewall will check the traffic against a list of known MAC addresses (see: IP/MAC binding)MACValidationPolicy
: can beaccept
ordrop
. Default isdrop
. Seeman shorewall.conf
for all valid valuesInterfaceRoleList
: list of network interface roles configurable from web interface. Default is:green,red,blue,orange
CheckIP
: comma-separeted list of IP monitored by LSM, to check if a provider is up or downMaxNumberPacketLoss
: number of maximum consecutive packets lost, over this threshold the provider is considered downMaxPercentPacketLoss
: percentage of maximum packet lost, over this threshold the provider is considered downPingInterval
: seconds between ICMP packets sent by LSM, default is5
NotifyWan
: can beenabled
ordisabled
, ifenabled
a mail is sent every time a provider changes its own stateNotifyWanFrom
: sender address for mails sent if NotifyWAN is set to enabledNotifyWanTo
: recipient address for mails sent if NotifyWAN is set to enabledVpnPolicy
: can bepermissive
orstrict
. If set topermissive
traffic between all VPNs (Rodwarrior OpenVPN, OpenVPN tunnels, IPSec tunnels) will be permittedSipAlg
can beenabled
(default) ordisabled
. It enables/disables the application level gateway (ALG) for SIP|H323 protocol (the netfilter conntrack kernel modules for these protocols)
Example
firewall=configuration
CheckIP=8.8.8.8,208.67.222.222
Docker=disabled
ExternalPing=enabled
HairpinNat=disabled
MACValidation=disabled
MACValidationPolicy=drop
MaxNumberPacketLoss=10
MaxPercentPacketLoss=50
NotifyWan=disabled
NotifyWanFrom=root@localhost
NotifyWanTo=root@localhost
PingInterval=5
Policy=permissive
SipAlg=enabled
TCLinklayer=
VpnPolicy=strict
WanMode=balance
Events¶
The main event for firewall configuration is firewall-adjust. The event contains a single action which fires the event named in the property event
inside the firewall
key into the configuration
database.
Other events:
- lokkit-save: base firewall implementation using lokkit
- nethserver-firewall-base-save: firewall implementation using Shorewall
- wan-uplink-update: fired when the status of an external interface changes
- conntrack-adjust: fired when we want to reset the conntrack and remove kernel modules used by shorewall
The wan-uplink-event
event takes at least two parameters:
- provider name: name of the provider involved
- action: can be
up
ordown
, describing the new provider status
Example:
signal-event wan-uplink-update down myisp
Policy¶
For every network packet traveling between firewall zones, the system will evaluate a list of rules to allow/block the specific traffic. Policies are default firewall rules which will be applied only if no other rule matches the ongoing traffic.
Firewall implements two standard policies:
- Permissive: will enable all traffic from green (loc) zone to red (net) zone.
- Strict: will block all traffic from green (loc) zone to red (net) zone. Permitted traffic should be explicitly allowed.
The firewall configures 4 default zones with built-in policies (see above). In the schema below, traffic is permitted from left to right and blocked from right to left:
GREEN -> BLUE -> ORANGE -> RED
To override a policy, you should create a firewall rule between zones.
IP/MAC binding¶
When MACValidation
option is enabled, the firewall analyzes all the traffic based on a well-known list of IPs associated to MAC addresses.
If the host generating the traffic is not inside the list, MACValidationPolicy
will be applied.
The list of IP/MAC association is created from DHCP reservations.
Thus, enabling MACValidation and leaving MACValidationPolicy set to drop, will block all traffic from hosts without a DHCP reservation.
Rules¶
Firewall rules can allow or deny traffic matching certain conditions.
Rules are saved inside the fwrules
database as records of type rule
.
Each rule record has the following fields:
key
: a unique key identifierPosition
: integer sorting keySrc
,Dst
: {literal*|*reference} where- literal is an IP, a CIDR,
any
(any source/destination) orfw
(the firewall itself) - reference has the form
prefix;value
, where prefix can be a DB type (host
,host-group
,zone
,iprange
,cidr
,mac
) or the stringrole
,value
is a DB key or an interface role name (green
,red
…).mac
objects are not supported inside theDst
field.
- literal is an IP, a CIDR,
Action
: can beACCEPT
,DROP
orREJECT
ACCEPT
allows the trafficREJECT
denies the traffic, an ICMP port unreachable packet is sent to the source addressDROP
discards the traffic without informing the source address (the connection will timeout)
Service
: (optional) can be a service object, a port number or a ndpi object. If a port number is used, both TCP and UDP protocols are matched.Time
: (optional) can be a time object, the rule will be enabled only if the time conditions is matchedLog
: can benone
orinfo
. If value isinfo
, all matched packets will be logged in/var/log/firewall.log
. Defaults tonone
status
: can beenabled
ordisabled
. Default isenabled
State
: (optional) select on which type of connection the rule will be applied to:new
or empty: default, the rule will be applied only to new connectionsall
: the rule will be applied to new and established/related connections
Description
: (optional)
Example of a rule accepting traffic:
1=rule
Src=host;myhost
Dst=192.168.1.2
Service=service;ssh
Action=accept
Position=32
Accept all traffic from myhost to myserver for ssh service (port 22):
db fwrules set 1 rule Src "host;myhost" Dst "host;myserver" Service ssh Action ACCEPT Log none status enabled Position 8765
Drop all traffic from 192.168.1.0/24 to 192.168.4.1 on TCP and UDP port 25:
db fwrules set 2 rule Src 192.168.1.0/24 Dst 192.168.4.1 Service 22 Action DROP Log none status enabled Position 5469
Template Fragment¶
Rules in the firewall can be added manually by a template fragment in the folder /etc/e-smith/templates/etc/shorewall/rules
For example drop a file 40YourSpecificRule
## 40nethvoice
{
my $iax = $nethvoice{'AllowExternalIAX'} || 'disabled';
my $webrtc = $nethvoice{'AllowExternalWebRTC'} || 'disabled';
if ($iax eq 'enabled') {
$OUT .= "# Enable IAX from red interfaces\n";
$OUT .= "?COMMENT Enable IAX from red interfaces\n";
$OUT .= "ACCEPT\tnet\t\$FW\tudp\t4569,5036\n";
}
if ($webrtc eq 'enabled') {
$OUT .= "# Enable WebRTC from red interfaces\n";
$OUT .= "?COMMENT Enable WebRTC from red interfaces\n";
$OUT .= "ACCEPT\tnet\t\$FW\ttcp\t8089\n";
}
$OUT .= "?COMMENT\n";
}
You can use all the settings below but you might be interested by the shorewall documentation also at http://shorewall.net/manpages/shorewall-rules.html)
\t
-> write a tab space (can be also written :$OUT .= "ACCEPT net $FW tcp 8089\n";)
ACCEPT
-> allows the trafficREJECT
-> denies the traffic, an ICMP port unreachable packet is sent to the source addressDROP
-> discards the traffic without informing the source address (the connection will timeout)REDIRECT
-> redirect the traffic to another firewall zone
The target may optionally be followed by “:” and a syslog log level (e.g, REJECT:info or Web(ACCEPT):debug).
loc
-> green (Local network)net
-> red (Internet network)blue
-> blue (Guest network)orang
-> orange (DMZ network)$FW
-> firewalltcp
-> tcp port (comma separated list of ports)udp
-> udp port (comma separated list of ports)
then you must expand your templates and restart your firewall by : signal-event firewall-adjust
Firewall objects¶
Firewall module uses objects to simplify rules creation. The use of objects is not mandatory but it’s strongly encouraged.
Supported objects are:
- Host
- Group of host
- Service
- CIDR
- Ip range
- Zone
- Time
- MAC address
A host is an already defined entry inside the hosts
db, or a new key of type host
:
name=host
IpAddress=IP
Description=
A host-group
is a group of hosts inside the hosts
db. Hosts in a host-group should always be reachable using the same interface.
For example: a group of host inside the LAN or on the Internet.
A host-group
db entry can be something like:
name=host-group
Members=host1,host2
A CIDR
is a group of hosts defined as a CIDR network. It’s saved inside the hosts
db:
mycidr=cidr
Address=192.168.100.0/24
Description=
A IP range
is a group of hosts defined as a range of IP. It’s saved inside the hosts
db:
myrange=iprange
Description=
End=192.168.100.20
Start=192.168.100.10
A zone represents a network zone which can be associated to an interface or a set of IP address. A zone
entry in networks
database can be something like:
name=zone
Network=CIDR
Interface=eth0
A configured network interface is automatically a zone.
A service can have a protocol and one or more ports. A service
entry in fwservices
database can be something like:
name=fwservice
Protocol=tcp/udp/tcpudp/icmp
Ports=port/port range
A service can also be a refence in the format ndpi;<protocol>
where protocol
is a supported protocol from nDPI kernel module.
To see a list of supported protocols:
db NethServer::Database::Ndpi keys
A time condition is a time
record entry in fwtimes
database.
All times are saved in local time and converted to UTC on template expansion.
Database example:
db fwtimes setprop officehours WeekDays 'Mon,Tue,Wed,Thu' TimeStart '09:00' TimeStop '18:00'
A MAC address is a mac
record entry inside macs
database.
The MAC must always have a Zone
property which specifies the network segment where the device is connected.
It’s something like:
mac1=mac
Address=52:54:00:05:2d:c3
Description=My mac test
Zone=green
Rules based on mac address¶
It’s possible to create rules based on MAC address only using a template-custom. For example to block internet access to a host on local network using its MAC address:
mkdir -p /etc/e-smith/templates-custom/etc/shorewall/rules
echo "DROP loc:~xx-xx-xx-xx-xx-xx net" > /etc/e-smith/templates-custom/etc/shorewall/rules/90mymac
Where xx-xx-xx-xx-xx-xx
is the MAC address to block.
See man shorewall-rules for more information.
Port forwarding¶
All port-forwards are saved inside the portforward
db.
Each record has:
key
: auto-increment idtype
: pfprotocol
: tcp/udpsrc
: can be a port number or a range in the form xxxx:yyyydst
: can be a port number, if empty the value ofsrc
is useddstHost
: destination host, can be an IP address or a hos firewall objectallow
: allowed ip address or network, see SOURCE at http://www.shorewall.net/4.2/manpages/shorewall-rules.htmlstatus
: enabled/disabledoriDst
: original destination ip, for example alias for a wan interface. If empty, the port forward is valid for all red interfacedescription
: optional description
Source NAT (sNAT)¶
All NAT one-to-one configurations are stored in networks
db.
Each value is a new attribute for an existing alias key and the name of attribute is FwObjectNat
that contains the reference of an associated host:
eth1:0=alias
FwObjectNat=host;host_name
ipaddr=11.11.11.11
netmask=255.255.255.0
role=alias
During template-expanding phase, the associated host is mapping with referenced IP and added in shorewall nat configuration. The file is /etc/shorewall/nat
.
More information are available here: http://shorewall.net/NAT.htm
Traffic shaping¶
Traffic shaping is implemented using Shorewall mangle and FireQOS: each mangle rule sets a well-known marker, markers are used to match traffic inside FireQOS tc classes.
The firewall needs to know how much inbound and outbound bandwidth has a red interface.
The bandwidth value (expressed in kbit) is stored inside FwInBandwidth
and FwOutBandwidth
properties, wich are
parts of the network interface record inside the networks
db.
Each red interface can have also the TCLinklayer
property, see FireQoS documentation supported values <https://firehol.org/fireqos-manual/fireqos-params-class/#linklayer-linklayer-name-ethernet-atm>.
FireQOS tutorial suggests to use 90% of the declared bandwidth to shape the inbound traffic faster.
On red interfaces with FwInBandwidth
and FwOutBandwidth
set, ethernet offloading is automatically disabled.
Example:
enp0s20f2=ethernet
FwInBandwidth=30000
FwOutBandwidth=24000
TCLinklayer=ethernet
bootproto=none
gateway=1.2.3.4
ipaddr=1.2.3.5
netmask=255.255.255.0
role=red
All traffic shaping rules are saved inside the fwrules
database with the same format.
Valid actions for traffic shaping rules are:
class;<name>
: set associated tc class.provider;<name>
: force the traffic to the provider specified byname
tc classess¶
tc classes are saved inside the tc
database with type class
.
Each tc class has the following properties:
BindTo
: empty (default) or comma-separated list of red interfaces. If one ore more interface is listed, the class is applied only to given interfaceDescription
: optional class description (used only in the UI)Mark
: integer value which identify the marker used for this class. Maximum is63
MaxInputRate
: maximum download rate, expressed in percentage of the total download bandwidthMaxOutputRate
: maximum upload rate, expressed in percentage of the total upload bandwidthMinInputRate
: reserved download rate, expressed in percentage of the total download bandwidthMinOutputRate
: reserved upload rate, expressed in percentage of the total upload bandwidthUnit
: bandwidth unit of measure for TC classes, default to%
, supported values are from FireQoS doc
Example:
high=class
BindTo=
Description=
Mark=2
MaxInputRate=
MaxOutputRate=
MinInputRate=10
MinOutputRate=10
Unit=%
low=class
BindTo=ens1
Description=
Mark=2
MaxInputRate=
MaxOutputRate=
MinInputRate=10
MinOutputRate=10
Unit=kbps
Assumptions and limitations¶
- All nDPI traffic is marked in forward chain. When a nDPI protocol is found, the whole connection is marked.
- Priority rules are in post chain and can use nDPI markers. If a priority rule uses a role (interface) as source, the rule can’t be added to postrouting chain since the packet is already natted: Shorewall will move the rule on top of forwarding chain.
- nDPI rules can’t block the http/https traffic if web proxy is enabled in transparent mode.
- All nDPI markers are read from
/proc/net/xt_ndpi/proto
and shifted by 8 bits. - Divert rules can’t be used with nDPI, because an established TCP connection can’t be moved between providers.
- Prerouting table is reserved by Shorewall for handlind the multi wan scenario.
See also:
Divert rules¶
A divert rule is used to force traffic to a specific provider.
For example, this rules will route all traffic to port 22 via the provider named myadsl:
1=rule
Src=192.168.1.0/24
Dst=0.0.0.0/0
Service=fwservice;ssh
Action=provider;myadsl
status=enabled
Position=2
Description=
Properties:
key
: numeric idSrc
: can be a ‘any’, role (execpt red), zone (not interface), host object, ip address, ip range or CIDRDst
: can be a zone (not interface), host object, ip address, ip range or CIDRAction
: provider object, in the form of “provider;<name>”Service
: (optional) can be a service objectstatus
: can be enabled or disabled. Default is enabledPosition
: integer sorting keyDescription
: (optional)
A rule is ignored during template expansion if:
- the source is red role
- the destination is a role which is not red
- source, destination and service are all set to any
- the provider doesn’t exists
- destination is set to any
Multi WAN¶
NethServer firewall can handle 15 red (WAN) interfaces. Implementation uses Shorewall with LSM (Link Status Monitor).
The LSM daemon takes care of monitoring WAN connections (interface) using ICMP traffic and it informs Shorewall about interface up/down events.
Each interface can be checked using multiple IPs (see checkip
property below). At least one IP must be reachable to mark the WAN connection as usable.
If no IP is specified (recommended option), the system will uses well-known default IPs (Google DNS and OpenDNS).
For each configured provider, LSM will send ping to a configured IP (checkip).
When a provider status changes, the system will signal a wan-uplink-update
event.
Inside the event, the action nethserver-shorewall-wan-update
invokes:
- shorewall enable <interface> when a red interface is usable
- shorewall disable <interface> then a red interface is not usable
When an interface is disabled, all associated routes will be deleted.
When a new TCP connection is started, a route is selected and all successive packets will always be routed via same interface. If the used interfaces goes down, the connection is closed.
Actually two behaviors are implemented: balanced and active-backup.
Balanced¶
All red interfaces are simultaneously used accordingly to the configured weight (see below).
Example:
Given a connection A with weight 2, and connection B with weight 1, the firewall will route a double number of connections via A over B.
Active-backup¶
Red interfaces are ordered using the configured weight: higher the weight, higher the route priority. The interface with maximum weight will be the active connection, all other interfaces will be used if the active one goes down.
Example
Given 3 wan connections:
- A with weight 3
- B with weight 2
- C with weight 1
All traffic is routed via A. On failure of A, all traffic is routed via B. When B goes down, C is used. Whenever A comes backup, all traffic is again routed through it.
Providers¶
Providers are an abstraction over red interfaces (see man shorewall-providers). All providers must have a weight which is used to select the route for packets.
A provider
record inside the networks
database has following properties:
key
: name of providerinterface
: associated red interface, it’s mandatoryweight
: weight of connection expressed with an integer number, it’s mandatoryDescription
: (optional) custom description
Example:
myisp=provider
interface=eth1
weight=5
Description=my fast provider
Multi WAN example¶
- Configure two interfaces as red, for example eth1 and eth2
db networks setprop eth1 role red
db networks setprop eth2 role red
signal-event interface-update
- Create two providers:
db networks set firstisp provider interface eth1 weight 2
db networks set secondisp provider interface eth2 weight 1
- Re-configure the firewall:
signal-event firewall-adjust
See /var/log/firewall.log
to check for up/down events.
Routes can be checked using:
shorewall show routing
Static routes¶
Static routes are saved inside the routes database with a record of type static. Example:
8.8.4.4=static
Description=My route
Mask=255.255.255.255
Router=89.97.220.225
Each record has the following properties:
key
: network addressMask
: network maskRouter
: gateway for the networkDescription
: a custom description (optional)
There is also a special type of static route called provider-static
.
These routes have the same properties as described above and are used to correctly route traffic for link monitor.
This type of rules should never be manually edited.
nethserver-firewall-migration¶
Migrate firewall configuration to NethSecurity: https://github.com/NethServer/nethsecurity
Migrate to another machine¶
Execute on NS7:
firewall-export
Generated file can be found at /var/lib/nethserver/firewall-migration/export.tar.gz
.
Access the NethSecurity installation and upload the export.tar.gz
archive using SSH,
then execute:
ns-import export.tar.gz
In-place migration¶
The firewall-migrate
procedure will requires about 400MB of free disk space.
The script will:
- prepare a NethSecurity image containing the firewall export archive
- write the image to the target disk
- reboot the system with the newly installed NethSecurity
At first boot, NethSecurity will automatically import the configuration.
Test the image build process:
- execute
firewall-migrate
- if no error occurs, verify the
/usr/share/nethserver-firewall-migration-builder/nethsecurity.img.gz
file exists
To start the migration process, pass the target device as first argument:
firewall-migrate /dev/vda
NOTE: The script will wipe the NethServer installation. The script does not ask for confirmation!
Cleanup¶
The downloaded image is preserved to speed up multiple firewall-migrate runs. To cleanup existing images and download a new one on next run, execute:
rm -rf /usr/share/nethserver-firewall-migration-builder/
nethserver-freepbx¶
This package configures FreePBX and Asterisk for NethServer MariaDB, Asterisk 13 and FreePBX 14 will be installed and configured.
FreePBX Web UI Access¶
FreePBX is now reachable at https://IP_ADDRESS/freepbx from green interfaces. To add an IP address or a network from red allowed to access interface, configure it from NethServer Web UI under “PBX Access” page
Backup¶
Since FreePBX configuration is stored in MariaDB, database dump are split inside data and configuration backup:
- configuration backup: contains the
asterisk
db with all configurations (extensions, trunks, etc.) - data backup: contanins the
asteriskcdrdb
db containg all PBX events
Open services from external networks¶
IAX and WebRTC access can be opened to external networks from server manager web gui -> PBX Access
SIP and other ports, from command line (Dangerous! Do it only if you know what you’re doing)
config setprop asterisk access public
signal-event firewall-adjust
Users¶
User are synced with NethServer users. You can manually force Userman module sync with
/usr/bin/scl enable rh-php56 "/usr/sbin/fwconsole userman sync"
WebRTC¶
You can create WebRTC extensions following this official guide from freepbx:
After the installation of Certificate Manager and WebRTC modules there is a valid self-signed certificate that can be used for WebRTC.
How to test it¶
Install “Certificate management” module
From “Advanced settings” page:
- Enable the mini-HTTP Server
- Set “SIP Channel Driver” to chan_pjsip
From “SIP settings”, inside the PJSIP tab, set the following values:
- Certificate Manager: default
- SSL Method: default
- Verify Client: No
- Verify Server: No
- udp: yes
- tcp: no
- tls: no
- ws: yes
- wss: yes
- Port listen: 5160
Configure a new PJSIP extension and set the following values:
- Enable AVPF: yes
- Enable ICE Support: yes
- Media use received transport: no
- RTP symmetric: yes
- Media encryption: DTLS-SRTP
- Require RTP (media) encryption: yes
- Enable DTLS: yes
- Use certificate: default
- DTLS verify: no
- DLTS setup: Act/Pass
- DTLS Rekey Interval: 0
Restart asterisk and temporarly disable the firewall:
systemctl restart asterisk shorewall clear
Open https://<server_ip>:8089/ws URL with the browser and accept the certificate
Try with one of the below WebRTC cients:
Known bugs¶
- WebRTC under Chrome is not allowed in HTTP
- WebRTC in UCP module on FreepBX may not work as expected. (http://community.freepbx.org/t/webrtc-phone-with-https/26698/9)
- WebRTC may not work on Firefox
Custom User Management configuration¶
The nethserver-freepbx-conf-users action
configures users using NethServer SSSD configuration.
This creates an entry in userman FreePBX module called NethServer [AD|LDAP]
.
If you need to edit this entry and you don’t want it to be modified when nethserver-freepbx-conf-users is launched again, change it’s name adding “Custom” (or any other string) at the end. Example: ‘NethServer AD’ -> ‘NethServer AD Custom’
If you remove NethServer [AD|LDAP]
string, another entry will be created by nethserver-freepbx-conf-users
action.
To check user synchronization, use this command:
/usr/bin/scl enable rh-php56 -- /usr/sbin/fwconsole userman --syncall --force --verbose
Syncronization need a secure connection, use SSL or enable STARTTLS
in Account Provider
configuration in NethServer Web GUI
Update from legacy OpenLDAP driver to OpenLDAP2 driver¶
Since nethserver-freepbx-14.0.5, if NethServer users are configured using OpenLDAP, FreePBX users are configured using FreePBX OpenLDAP 2 driver instead of legacy one. If you have installed nethserver-freepbx before 14.0.5, and your user provider is configured using LDAP, you’re using legacy driver. You can check in FreePBX User Manager module interface if NethServer LDAP driver is “OpenLDAP Directory (Legacy)”
Updating from legacy driver to the new one, allows to permit access to FreePBX interface and UCP to LDAP users, but migration isn’t automatical because users would lose default extension associated and other custom options. The openldap_migration_from_legacy script, does the driver migration and restore users default_extensions. Other custom users options could be lost anyway. To execute migration, launch:
/usr/src/freepbx/openldap_migration_from_legacy
Cockpit API¶
settings/read¶
This api returns asterisk
and httpd-fpbx
configuration.
Input¶
config
:asterisk
orhttpd-fpbx
Input example (asterisk)¶
{
"config": "asterisk"
}
Output example (asterisk)¶
{
"configuration": {
"type": "service",
"name": "asterisk",
"props": {
"status": "enabled",
"TCPPorts": "5060,5061,5038,8088,8089",
"AllowExternalIAX": "disabled",
"access": "green",
"AllowExternalSIPS": "enabled",
"AllowExternalWebRTC": "disabled",
"UDPPorts": "4569,5036,5060,5160,10000:20000"
}
}
}
Input example (httpd-fpbx)¶
{
"config": "httpd-fpbx"
}
Output example (httpd-fpbx)¶
{
"configuration": {
"type": "service",
"name": "httpd-fpbx",
"props": {
"status": "enabled",
"access": "green",
"ValidFrom": "10.20.30.40/255.255.255.0,10.0.0.5/255.255.255.0"
}
}
}
settings/validate¶
This api validates an IP address allowed to access PBX web interface
Input¶
ipAddress
: an IP addressnetmask
: a network netmask
Input example¶
{
"ipAddress": "10.20.30.40",
"netmask": "255.255.255.0"
}
Output example¶
{
"state": "success"
}
settings/update¶
This api updates asterisk
and httpd-fpbx
configuration.
Input¶
config
:externalAccess
orwebInterfaceAccess
allowExternalIAX
: boolean flag to enable or disable external telephony access through IAX protocol (only ifconfig
:externalAccess
)allowExternalSIPS
: boolean flag to enable or disable external telephony access through secure SIP TLS protocol (only ifconfig
:externalAccess
)webInterfaceAccess
: list of IP addresses and netmasks allowed to access PBX web interface (only ifconfig
:webInterfaceAccess
)
Input example (externalAccess)¶
{
"config": "externalAccess",
"allowExternalIAX": "enabled",
"allowExternalSIPS": "enabled"
}
Input example (webInterfaceAccess)¶
{
"config": "webInterfaceAccess",
"webInterfaceAccess": [
{
"ipAddress": "5.6.7.8",
"netmask": "255.255.255.0"
},
{
"ipAddress": "10.20.30.40",
"netmask": "255.255.255.0"
}
]
}
nethserver-hotsync¶
What is it ?¶
HotSync aims to reduce downtime in case of failure, syncing your NethServer with another one, that will be manually activated in case of master server failure.
Normally, when a hardware damage occurs, the time needed to restore service is:
- fix or buy another server: from 4h to 2 days
- install the OS: 30 minutes
- restore backup: from 10 minutes to 8 hours
In summary, users are able to start working again with data from the night before failure after a few hours/days. Using HotSync, time 1 and 3 are 0, 2 is 5 minutes (time to activate spare server). Users are able to start working again in few minutes, using data from a few minutes before the crash.
By default all data included in backup are synchronized every 15 minutes. MariaDB databases are synchronized too, unless databases synchronization isn’t disabled. Applications that use PostgreSQL are synchronized (Mattermost, Webtop5) unless databases synchronization isn’t disabled.
Requirements¶
Two server machines are needed:
- MASTER: active server on production
- SLAVE: warm backup server
For a correct restore, it’s suggested to configure HotSync on two identical servers or two servers with same network cards number, name and position. If the master and slave servers differ, the restore procedure may behave unexpectedly (see Troubleshooting section on NethServer docs).
How it works¶
MASTER machine is the currently production server and it makes a backup in the SLAVE machine every “n” minutes. When a crash occurs on the master, slave becomes active taking the same IP address of the master. Doing so the downtime is minimal, only the time to launch some commands on the SLAVE.
Installation¶
On both MASTER and SLAVE, install nethserver-hotsync:
# yum install nethserver-hotsync --enablerepo=nethforge
Configuration¶
MASTER configuration:
[root@master]# config setprop rsyncd password <PASSWORD>
[root@master]# config setprop hotsync role master
[root@master]# config setprop hotsync SlaveHost <SLAVE_IP>
[root@master]# signal-event nethserver-hotsync-save
SLAVE configuration:
[root@slave]# config setprop rsyncd password <PASSWORD>
[root@slave]# config setprop hotsync role slave
[root@slave]# config setprop hotsync MasterHost <MASTER_IP>
[root@slave]# signal-event nethserver-hotsync-save
The <PASSWORD> must be the same on both master and slave.
If MySQL or PostgreSQL are installed, they will be synchronized by default. To disable databases synchronization:
[root@master]# config setprop hotsync databases disabled
[root@master]# signal-event nethserver-hotsync-save
You can update these settings also from Cockpit interface.
How to restore¶
The following procedure puts the SLAVE in production when the master has crashed.
switch off MASTER
if the SLAVE machine must run as network gateway, connect it to the router/modem with a network cable
following command changes the IP address and cuts off the network connection. If you are connected through an ssh console, launch the
screen
command[root@slave]# screen
on SLAVE launch command, and read carefully its output
[root@slave]# hotsync-promote
If no Internet connection is detected (e.g. you are restoring a firewall on a machine that was passing through crashed master for Internet connection), the scripts will purpose you some options:
- Restore master network configuration (IMPORTANT: use this option only if two servers are identical - NIC number, names and positions must be identical)
- Fix network configuration from Cockpit GUI (when restoring on different hardware)
- Continue without internet: assign correct roles before proceed with this option. Some events could fails (not recommended)
If necessary go to Server Manager or Cockpit GUI, in page Network and reassign roles to network interfaces as master one. Remember also to recreate bridge if you have configured DC. In case of DC errors consult troubleshooting section before proceed with network restore.
launch command
[root@slave]# /sbin/e-smith/signal-event post-restore-data
if an USB backup is configured on MASTER, connect the backup HD to SLAVE
How to restore original server¶
To put again in production original crashed server:
configure it as SLAVE
synchronize it
switch off current MASTER
from bash execute:
# hotsync-slave
restore configuration backup following above instructions
How to synchronize custom paths¶
It is possible to customize HotSync adding all kind of data through the use of plugins. The directory of plugins is /etc/hotsync.d. Executable files in this directory, are executed before the synchronization.
INCLUDE_FILE and EXCLUDE_FILE are the files that contain the list of paths to include and to exclude to/from HotSync. Those two variables are passed as arguments to the scripts of this directory when they are executed.
If you want to add files to the synchronization, append them to the INCLUDE_FILE. Append them to EXCLUDE_FILE to remove from the synchronization.
How to force packages installation on SLAVE¶
“hotsync-slave” script extracts from MASTER configuration backup the list of packages to install and install them. You can force the operation executing bash command:
# hotsync-slave
Components¶
hotsync¶
is a shell script launched by cron every 15 minutes
uses a lockfile to ensure that only one instance at a time is executed
uses secure communication with rsync over stunnel
creates a list of files to be included and another one to be excluded from rsync (using backup-data configuration and hotsync own logic)
launch backup on MASTER
launch an rsync that copy listed files from MASTER to SLAVE in a secure manner using stunnel
if something fails, root is notified with an email
you can check files that will be copied on the next synchronization using the command:
hotsync --dry-run
hotsync-slave¶
Automatically executed on SLAVE every 60 minutes, extracts from MASTER configuration backup the list of packages to install and install them.
Supported packages¶
- nethserver-antivirus
- nethserver-backup-config
- nethserver-backup-data
- nethserver-base
- nethserver-c-icap
- nethserver-cockpit
- nethserver-collectd
- nethserver-cups
- nethserver-dante
- nethserver-dc
- nethserver-dedalo
- nethserver-directory
- nethserver-dnsmasq
- nethserver-duc
- nethserver-ejabberd
- nethserver-evebox
- nethserver-fail2ban
- nethserver-firewall-base
- nethserver-freepbx > 14.0.3
- nethserver-httpd
- nethserver-hylafax
- nethserver-iaxmodem
- nethserver-ipsec-tunnels
- nethserver-janus
- nethserver-letsencrypt
- nethserver-lightsquid
- nethserver-mail
- nethserver-mattermost
- nethserver-mysql
- nethserver-ndpi
- nethserver-netdata
- nethserver-nextcloud
- nethserver-ntopng
- nethserver-nut
- nethserver-openssh
- nethserver-openvpn
- nethserver-pulledpork
- nethserver-restore-data
- nethserver-roundcubemail
- nethserver-samba
- nethserver-samba-audit
- nethserver-squid
- nethserver-squidclamav
- nethserver-squidguard
- nethserver-sssd
- nethserver-subscription
- nethserver-suricata
- nethserver-vpn-ui
- nethserver-vsftpd
- nethserver-webtop5 (z-push state is not synchronized)
Packages nethserver-ntopng and nethserver-evebox are reinstalled without migrating history.
Please read the docs before proceed with restore.
nethserver-httpd¶
NethServer tries to remain compliant with the upstream configuration for httpd
.
In Apache 2.4 every global option is inherit by all virtual hosts, except for the Rewrite directives.
In a clean installation the only defined virtual host is the one on port 443.
When nethserver-httpd-virtualhosts is installed, the package adds a new default virtual host.
This virtual host includes the /etc/httpd/conf.d/default-virtualhost.inc
file
to mimic the upstream behavior.
See also “Certificate management” and “Virtualhosts” section for further information.
Templates¶
Available templates under /etc/e-smith/templates/
directory:
etc/httpd/conf.d/default-virtualhost.inc
: common configuration included inside default virtual host on port 80etc/httpd/conf.d/nethserver.conf
: default httpd configuration, includesconf.d/default-virtualhost.inc
. Normally a package shouldn’t add a fragment to it.etc/httpd/conf.d/welcome.conf
: configures the “Welcome” page if there is no default index page for the root URLhttpd/vhost
: everything inside will be included indefault-virtualhost.inc
. DO NOT USE, it’s here only for backward compatibility
Web applications¶
Every package which needs to change the Apache configuration:
- should add a static (or template-generated) file inside
/etc/httpd/conf.d/
directory - must include all Rewrite options inside a fragment for
default-virtualhost.inc
Example¶
Package named nethserver-mywebapp
which requires a rewrite rule from http to https.
The web application must be accessible under http://<server_address>/mywebapp
.
Static file /etc/httpd/conf.d/mywebapp.conf
:
Alias /mywebapp /usr/share/mywebapp
<Directory "/usr/share/mywebapp">
AllowOverride None
Options None
Require all granted
</Directory>
Rewrite rule fragment /etc/e-smith/templates/etc/httpd/conf.d/default-virtualhost.inc/30mywebapp
:
#
# 30mywebapp
#
RewriteEngine on
RewriteRule ^/mywebapp(/.*)?$ https://%\{SERVER_NAME\}/mywebapp$1 [L,R=301]
Inside the createlinks
file, the configuration should be applied during the update event:
my $event = "nethserver-mywebapp-update";
event_templates($event,
'/etc/httpd/conf.d/default-virtualhost.inc',
);
event_services($event,
'httpd' => 'reload',
);
configuration database¶
httpd=service
SSLCipherSuite=DEFAULT:!EXP:!SSLv2:!DES:!IDEA:!SEED:+3DES
TCPPorts=80,443
access=green,red
status=enabled
XForwarded=disabled
XForwarded
: this setting forwards to all the proxypass vhosts the full https scheme and not only host header (in X-Forwarded-Host). It is a limitation of mod_proxy, the virtualhost on the proxy apache must transmit the original information of the request.
vhost database¶
A new vhost
database is defined by this module. It contains records of type
vhost
, similar to:
vh1=vhost
Access=private
Description=description_text
ForceSslStatus=enabled
FtpPassword=ftp_password_value
FtpStatus=enabled
PasswordStatus=enabled
PasswordValue=http_password_value
ServerNames=www.nethserver.org,www.example.com
SslCertificate=/etc/pki/tls/certs/NSRV.crt
status=enabled
PhpRhVersion=default
PhpCustomSettings=disabled
AllowUrlfOpen=enabled
MemoryLimit=512
UploadMaxFilesize=4
PostMaxSize=8
MaxExecutionTime=0
MaxFileUploads=20
The database contains a special default
record which represents the default
virtual host:
default=vhost
Description=Default virtual host
FtpPassword=
FtpStatus=enabled
This virtual host is always enabled and can’t be deleted. If FTP access is
enabled, the user will be chrooted inside /var/www/html
directory.
proxypass database¶
The proxypass database contains records of type
VhostReverse
, similar to:
ns7loc7.org=VhostReverse
AllowEncodedSlashes=disabled
CertVerification=no
Description=
HTTP=yes
HTTPS=yes
PreserveHost=yes
SslCertificate=
Target=http://localhost:8000
ValidFrom=
WebSockets=disabled
WebSocketsPath=
AllowEncodedSlashes
(enabled/disabled) : Some web applications may expect encoded path separators (%2F
for/
) in the request URL. Apache does not accept the path separator in the encoded form. Set this prop to enabled if the encoded form can be safely forwarded to the backend (see Target) URL.CertVerification
(yes/no) : If the Target URL has the https scheme, accept its certificate even if it is not valid.PreserveHost
(yes/no): When enabled, this option will pass the HTTPHost
header line from the incoming request to the proxied host, instead of thehostname
specified in the Target URL field.WebSockets
(enabled/disabled) : The WebSocket allows to open a two-way interactive communication session between the user’s browser and the server. TheWebSocketsPath
is optional.Target
: The URL where the original request is forwarded. An URL has the form<scheme>://<hostname>:<port>/<path>
.HTTPS
andHTTP
(yes/no): If enabled, the URL path or virtual host name can be accessed only with an SSL/TLS connection.SslCertificate
: Select a certificate that is compatible with the virtual host name. Leave empty to use the default SSL certificate.ValidFrom
: Restrict the access from the given list of CIDR networks. Elements must be separated with a,
(comma).
ProxyPass
, similar to:
folder=ProxyPass
Description=
HTTP=yes
HTTPS=yes
Target=http://localhost:8000
ValidFrom=
HTTPS
andHTTP
(yes/no): If enabled, the URL path or virtual host name can be accessed only with an SSL/TLS connection.ValidFrom
: Restrict the access from the given list of CIDR networks. Elements must be separated with a,
(comma).Target
: The URL where the original request is forwarded. An URL has the form<scheme>://<hostname>:<port>/<path>
.
rh PHP software collection¶
With the new cockpit server-manager the PHP version can be modified
inside the virtualhost panel. It installs a pool of PHP-FPM, this new php
version is only relevant to the current apache virtualhost. The prop
PhpRhVersion
is used to set the PHP version (default
is the PHP 5.4,
php71
,php72
,php73
).
The documentation page of the project are: - https://www.softwarecollections.org/en/scls/rhscl/rh-php71/ - https://www.softwarecollections.org/en/scls/rhscl/rh-php72/ - https://www.softwarecollections.org/en/scls/rhscl/rh-php73/
If the prop PhpCustomSettings
is set to disabled
the PHP setting values
are inherited from the default values of PHP (from esmith configuration database), if
enabled
each vhost gets its own PHP values from its vhost props.
Events¶
signal-event nethserver-httpd-update
signal-event nethserver-httpd-save
NethServer 6 upgrade¶
Shared folders from NethServer 6 with property HttpStatus
set to enabled
can
be migrated to virtual hosts using the vhost-migrate-ibay
event:
signal-event vhost-migrate-ibay <ibay-name>
If the original ibay was availble to any virtual hosts (HttpVirtualHost = __ANY__
),
the ibay will be migrated inside the default
virtual host.
Otherwise a new virtual host record will be created.
The migration process is also available from the web interface.
UI plugins¶
The Modify action can be extended to display additional tabs, by adding a
controller and the respective template under ModifyPlugin/
directories.
See the Samba User plugin on NethServer 6.x as an example
Welcome page¶
If there is no index page for the root URL, a default welcome page is shown accessing the HTTP and HTTPS ports of the server.
If Cockpit UI is installed, the welcome page displays a customizable app launcher; users can choose which apps to show in the launcher by accessing Cockpit Applications page and clicking the Add to home page
command in the kebab menu of any app.
It is possible to display an alternative welcome page that replaces the app launcher:
create a subdirectory inside
/usr/share/httpd/noindex/
, e.g.mywebsite
put a custom index page (e.g. index.html) inside
/usr/share/httpd/noindex/mywebsite/
create a subdirectory
res
inside/usr/share/httpd/noindex/mywebsite/
put all the assets used by your page (images, scripts, …) inside
/usr/share/httpd/noindex/mywebsite/res/
the assets can be accessed from the index page using the prefix
/res_mywebsite/
, e.g.<link rel="stylesheet" type="text/css" href="/res_mywebsite/style.css">
make sure the primary page is called
index.html
execute the following commands:
config setprop httpd HomePage mywebsite signal-event nethserver-httpd-update
your custom welcome page is now accessible the the HTTP and HTTPS ports
If you want to switch back to the default app launcher page execute:
config setprop httpd HomePage nethserver
signal-event nethserver-httpd-update
nethserver-httpd-admin¶
Apache configuration for the web administration console
This Apache instance listens SSL connections on a non-standard port (by default 980) and execute system commands through sudo
.
The main PHP scripts (controllers) are located in /usr/share/nethesis/nethserver-manager/
directory.
Apache mod_rewrite
configuration hides index.php
in URLs. A generic production URL has the following form: https://<hostname>:980/<lang>/<ModuleId>
An additional controller index_dev.php
is available for development.
It produces more verbose logging, uses uncompressed JS and CSS libraries, and shows clear text names of widget target names.
The URL form for development is: https://<hostname>:980/index_dev.php/<lang>/<ModuleId>
To enable index_dev.php
controller, you need to execute the following command:
# touch /usr/share/nethesis/nethserver-manager/debug
smwingsd daemon¶
To speed up database read operations, avoiding sudo
calls, the smwingsd
Perl daemon is contacted.
As httpd-admin
, the daemon is controlled by systemd, and on start reads configuration from /etc/smwingsd.conf
.
It listens on the local Unix socket /var/run/smwingsd.sock
. You can contact it with a command like this:
# php -r '$payload=json_encode(array_slice($argv, 1), TRUE); print pack("CN", 0x10, strlen($payload)) . $payload;' configuration getjson sysconfig \
| nc -U /var/run/smwingsd.sock \
| cut -b 6- \
| python -mjson.tool
{
"name": "sysconfig",
"props": {
"Copyright": "",
"ProductName": "NethServer",
"Registration": "none",
"Release": "rc1",
"Version": "6.5"
},
"type": "configuration"
}
Customization¶
You can customize some parts of the NethServer web interface:
- logo
- favicon
- colors
- top bar background image
- menu bar background image
Colors¶
Simple configuration database entries for a theme green and red:
config setprop httpd-admin colors "green,red,#1d247c"
Reload the interface, colors should be applied respectively on: top header, left menu, text headers (also in tables and links on left menu)
Logo and favicon¶
Copy a custom logo (best size: 185 x 30) inside /usr/share/nethesis/nethserver-manager/images/ directory. Change the default logo with this command:
config setprop httpd-admin logo mylogo.png
Copy a custom favicon inside /usr/share/nethesis/nethserver-manager/images/ directory. The favicon can be in PNG or ICO format, recommended sizes are 16x16 or 32x32 pixels. Change the default logo with this command:
config setprop httpd-admin favicon myfavicon.png
Reload the interface and check the new logo and favicon are loaded.
Note
Favicons are often cached by browsers: be patient and reload the page after a little while.
Database example¶
httpd-admin=service
ForcedLoginModule=
MaxSessionLifeTime=
MaxSessionIdleTime=
SSL=enabled
TCPPort=980
access=green,red
colors=
favicon=favicon.png
headerBackground=
logo=
menuBackground=
status=enabled
Session timeouts¶
The Server Manager session validity is subjected to forced logoff under the following conditions
- after an idle/inactivity period. It occurs when no user activity is detected in a certain amount of time
- after the maximum session lifetime has elapsed. It occurs if the user does not log out voluntarily
To configure the timeouts (both expressed as seconds) run the following command:
config setprop httpd-admin MaxSessionIdleTime 900 MaxSessionLifeTime 86400
To disable the session timeouts, set the values to 0
or the empty string:
config setprop httpd-admin MaxSessionIdleTime '' MaxSessionLifeTime ''
The new timeout values will affect new sessions. They does not change any active session.
nethserver-hylafax¶
Hylafax is a fax server for receiving and sending faxes.
Key features:
- Modem automatic configuration
- Virtual modem
- Ingoing/outgoing fax configuration
- Extensible accounting system
- Mail2Fax
- Mail server integration
Modem configuration¶
Hylafax+ uses a tool called faxaddmodem
which guides the user throw modem configuration. We developed a simple non-interactive wrapper called neth-addmodem
.
The script reads basic information, like the fax device name, from the SME db and use these data to setup the modem.
Accounting (FaxAccounting)¶
Every time a fax is received or sent, Hylafax will execute the /var/spool/hylafax/etc/FaxAccounting
script. FaxAccounting runs all executable scripts inside the directory /var/spool/hylafax/etc/accounting
in alphabetical order.
The default configuration will execute the script 00savefiles
which saves all submitted and received faxes to /var/lib/nethserver/fax/docs
Incoming fax (FaxDispatch)¶
Hylafax requires a mail address where incoming faxes will be sent.
If SendTo
prop is not set, all faxes will be forwarded to root mailbox.
When a fax is received, Hylafax will execute the /var/spool/hylafax/etc/FaxDispatch
script. FaxDispatch runs all executable scripts inside the directory /var/spool/hylafax/etc/dispatch
in alphabetical order.
The default configuration will execute the script 90print
. If hylafax
prop PrintReceived
is enabled, all faxes are printed into the PrinterName
printer.
Outgoing fax (FaxNotify)¶
When a fax is sent, Hylafax will execute the /var/spool/hylafax/etc/FaxNotify
script. FaxNotify runs all executable scripts inside the directory /var/spool/hylafax/etc/notify
in alphabetical order.
Custom scripts¶
When adding a custom script not handled by an RPM, please add the script in a directory inside /var/lib/nethserver/fax/
and link it to the final destination. In fact the /var/spool/hylafax/etc/
directory is not (actually) in the backup.
So, for example, if you wish to add a 90test
script to accounting, save the files in the /var/spool/hylafax/etc/accounting
directory and link to the Hylafax configuration directory:
ln -s /var/lib/nethserver/fax/accounting/90test /var/spool/hylafax/etc/accounting/90test
Authentication¶
Clients are authenticated using reverse dns queries: all ip resolved with the same domain as server itself are automatically authenticated. If reverse dns query, client can authenticate using user name and password of a user inside faxmaster
group.
Clients should always use username, despite of authentication method used.
Internationalization¶
Notification on incoming and outgoing faxes can be localized using the Lang
property. Default value is en_US
.
Client configuration¶
The only tested client is YajHFC (see http://www.yajhfc.de/). Current configuration works only with LAN clients.
Configuration:
- Insert server ip (or hostname) and username
- Leave blank the password field
- Disable passive mode
Configuration DB¶
Example:
hylafax=configuration
AreaNumber=
ClientShowReceived=disabled
CountryCode=39
DateFormat=DMY
Debug=disabled
DispatchFileType=ps
FaxDevice=ttyS0
FaxName=fax
FaxNumber=
InternationalPrefix=00
Lang=it_IT
LongDistance=0
Mail2Fax=disabled
Mode=both
NotifyFileType=tif
NotifyMaster=always
PBXPrefix=
PaperSize=A4
PrintReceived=disabled
PrinterDriver=ljet4
PrinterName=
Resolution=196
RingsBeforeAnswer=1
SambaFax=disabled
SambaFaxName=SambaFax
SendReport=disabled
SendTo=aa``test.tld
SummaryReport=disabled
WaitDialTone=enabled
hylafax-hfaxd=service
TCPPort=4559
status=enabled
access=private
hylafax-faxq=service
status=enabled
nethserver-iaxmodem¶
The module uses modems
database to store record describing a iax modem.
After a new record is created, the system will generate all configuration files in /etc/iaxmodem
using the action nethserver-iaxmodem-modemsetup
.
When a modem is selected under the nethserver-hylafax web ui, the action nethserver-iaxmodem-hylasetup
will create a sutiable modem configuration file under /var/spool/hylafax/etc
directory.
Multiple IAX modems¶
Current nethserver-hylafax implementation doesn’t handle more than one modem. To use multiple IAX modems:
Create a IAX from web UI
Copy an existing
config.IAXxx
file and change number and identification strings Eg:cp /var/spool/hylafax/etc/config.ttyIAX351 /var/spool/hylafax/etc/config.ttyIAX350
Copy an existing faxgetty configuration, and change the device name inside the new file: Eg:
cp /etc/init/faxgetty-ttyIAX350.conf /etc/init/faxgetty-ttyIAX351.conf sed -i 's/ttyIAX350/ttyIAX351/g' /etc/init/faxgetty-ttyIAX350.conf
Start services
start faxgetty-ttyIAX351 service hylafax restart
Configuration database:
mymodem=iax
cidName=NethServer
cidNumber=351
extension=351
password=351
server=192.168.1.1
nethserver-ipsec-tunnels¶
This pacakge provides the configuration of VPN tunnels based using IPSec. The implementation can support custom properties to override the configuration from web interface.
All records of type ipsec-tunnel
are saved insided the vpn
database.
Every property in the form Custom_<name>
will override any existing property.
The same syntax can also be used to set any IPsec options supported by OpenSwan.
Example: override left prop
Given the following record:
nethesis-test=ipsec-tunnel
compress=no
dpdaction=hold
esp=auto
ike=auto
left=192.168.2.246
leftid=@nethesis
leftsubnets=192.168.1.0/24
pfs=yes
psk=Nethesis,12345678911
right=1.2.3.4.5
rightid=@test
rightsubnets=192.168.6.0/24
status=enabled
The admin can override the left property:
db vpn setprop nethesis-test Custom_left %any
signal-event nethserver-ipsec-save
Example: set new option
Set aggressive mode:
db vpn setprop nethesis-test Custom_aggrmode yes
signal-event nethserver-ipsec-save
nethserver-janus¶
This package configure Janus Gateway https://github.com/meetecho/janus-gateway
Start Janus Gateway at boot¶
To start Janus Gateway and enable it at boot, execute:
config setprop janus-gateway status enabled
signal-event nethserver-janus-update
NAT modes¶
Janus-gateway can operate in three different NAT mode:
- STUN (default)
- ICE
- 1:1 NAT
There are 4 prop under janus-gateway
key to configure NAT mode and options:
NatMode
<stun|ice|1:1>StunServer
is the STUN server to use. Default is stun1.l.google.com. Ignored if mode isn’t stunStunPort
is the STUN server port. Default is 19302. Ignored if mode isn’t stunPublicIP
is the public IP address of the server that runs janus-gateway . Ignored if mode isn’t 1:1
For configuring TURN, create a template custom
STUN and ICE enforced interfaces¶
You can choose which interface should be used for the ICE candidates gathering. In default configuration Janus try to use all interfaces, except ‘vmnet*’, that is VMware interface which is known to cause problems, ‘tun*’ and ‘tap*’ interfaces for VPNs, ‘virb*’ for KVM and ‘vb-*’ for NethServer AD container. You can modify this behavior by explicit list interfaces to use or by listing interfaces to exclude. You can configure one of those two props:
ICEEnforceList
comma separated list of interfaces to use for ICE gathering. For instances “e0,e1”.ICEIgnoreList
comma separated list of interfaces to exclude from ICE gathering. Janus will use all interfaces except those. For instance ‘e3,vmnet,10.0.0’.
Disable plugins¶
All plugins are enabled by default. To disable some of them execute:
config setprop janus-gateway DisabledPlugins "<PLUGINS_NAMES_SEPARATED_BY_COMMAS>"
signal-event nethserver-janus-update
Example:
config setprop janus-gateway DisabledPlugins "libjanus_voicemail.so,libjanus_recordplay.so"
signal-event nethserver-janus-update
To re-enable them:
config setprop janus-gateway DisabledPlugins ""
signal-event nethserver-janus-update
Debug problems with Admin Api¶
The Janus Admin Api are used to debug problems (e.g. ssl, ice, …).
nethserver-janus
automatically enables Admin Api on HTTP localhost
on port 7088
.
To use it:
make an ssh tunnel:
ssh IP -L 7088:localhost:7088
extract the Admin Api secret:
grep admin_secret /opt/janus/etc/janus/janus.jcfg | cut -d ' ' -f 3
Follow the official documentation on how to use admin api: https://janus.conf.meetecho.com/docs/admin.html
Note: here is an example on how to use it with Node.js: https://gist.github.com/alepolidori/720e47448e82b99ef3495a1849773ecc
nethserver-lang¶
The localization process turns around the Transifex online tool.
The Transifex client, available as /usr/bin/tx
command, is required. Some
Linux distributions have already packaged it. For NethServer, it is
available from EPEL.
yum --enablerepo=epel install transifex-client
yum install git
Edit ~/.transifexrc
:
[https://www.transifex.com]
hostname = https://www.transifex.com
password = <YOURPASSWORD>
token =
username = <YOURUSERNAME>
Refer also to Internationalization in the Developer’s Manual.
Building a release RPM¶
Periodic builds of the RPM are started automatcally at Travis CI. See nethserver-makerpms in the Developer’s Manual for more information about the automated builds on Travis CI.
According to the .travis.yml
and build.sh
scripts, the build procedure pulls translations
of existing catalogs from Transifex, creates, adds a release tag and pushes a git commit and tag.
Then starts the build of a new RPM that is published in the updates
repository.
As alternative, a release commit can be manually prepared with the :releasetag-section: command, as for
other nethserver-*
packages.
$ releasetag -T X.Y.Z
...edit the changelog entry
$ git push --follow-tags
The manual procedure is still useful when new resources are added to .tx/config
.
See the Adding new resources section below.
Building RPMs for testing¶
To manually build the RPMs for testing prepare the build environment:
Install a build environment as described in Building RPMs.
(Fork and) Clone the git repository into your machine:
git clone https://github.com/NethServer/nethserver-lang.git
Then run this workflow with nethserver-makerpms:
Pull updates from Transifex:
tx pull -a
(optional) clean up partially translated languages:
git add <files for new configurations just added to .tx/config> git clean -f -d
Commit changes to git repository:
git commit -a -m 'Pulled translation strings from Transifex'
Build RPMs:
makerpms nethserver-lang.spec
If everything is ok, do not forget to push the commit back to GitHub, and open a Pull Request:
git push
Adding new resources¶
After the developer has pushed sources to Transifex,
the new resources must be added here, inside the .tx/config
file.
For Cockpit resources, refer to https://nethserver.github.io/nethserver-cockpit/i18n/#translations-packages
For for PHP resources add to .tx/config
a section like this
[nethserver.MyModule]
file_filter = locale/<lang>/server-manager/NethServer_Module_MyModule.php
source_file = locale/en/server-manager/NethServer_Module_MyModule.php
source_lang = en
type = PHP_ALT_ARRAY
Then retrieve all files:
tx pull -r nethserver.MyModule -a
Finally, add all files to the repository:
git add *NethServer_Module_MyModule*
And push the commit and open a Pull Request on GitHub
git push
nethserver-lightsquid¶
LightSquid, Web Proxy report
LightSquid is a lite and fast log analizer for squid proxy.
It parses /var/log/squid/access.log
log file on daily basis and generate
an HTML report.
LightSquid is enabled by default when the nethserver-lightsquid package is installed.
Web interface can be accessed at http://<server>/<hash>
.
The hash is auto-generated (see below).
Database¶
The configuration is saved in the lightsquid
key inside the configuration
database.
Example:
lightsquid=configuration
alias=44da17a293c2aae17815bb95af98883355520aef
nethserver-mail¶
Mail system implementation based on Postfix, Dovecot, Rspamd, OpenDKIM. The mail system configuration is splitted into many RPMs, described in the following sections.
- nethserver-mail-common
- nethserver-mail-smarthost
- nethserver-mail-disclaimer
- nethserver-mail-filter
- nethserver-mail-server
- nethserver-mail-ipaccess
- nethserver-mail-getmail
- nethserver-mail-p3scan
- mail-quarantine
- Database format
- Mail quota
- Disabled users
- Testing Dovecot with Mutt
- Set special ACL on mailboxes
- IP-based IMAP access restriction
- Enable dovecot IMAP rawlog
- Access the rspamd UI
- Bayesian rules upgrade to rspamd
- Sender address validation
- Postfix SMTP listening ports
- Dynamic group aliases
nethserver-mail-common¶
- Common infrastructure for
nethserver-mail-server and nethserver-mail-filter
, Postfix-based. - Relay
- Queue parameters: age + message size
- MX record configuration
nethserver-mail-smarthost¶
- Send mail through the given MTA (smarthost), with SMTP/AUTH
- StartTLS encryption
- Set sender address for mail sent from root user (see Notifications section under nethserver-base package README)
nethserver-mail-disclaimer¶
- Attach disclaimer/legal notice to outbound messages for certain domains
- Runs
altermime
with Postfixcontent_filter
option
nethserver-mail-filter¶
- Based on Rspamd
- Anti-spam with DNSBL (see: nethserver-unbound)
- Anti-virus
- Attachment block
- Real-time Blackhole List (RBL)
- Sender Policy Framework (SPF)
- Customized spam threshold
- Sender WBL, Recipient whitelist
nethserver-mail-server¶
- IMAP/POP3 mailbox access protocols
- STARTTLS enabled by default
- Mail quota
- Sieve filters
- Postfix/dovecot-lda integration
- Multi-domain
- Domain-specific configuration
- Pseudonyms
- SMTP authentication
- Active Directory integration
- SpamAssassin’s Bayesian classifier training (
spamtrainers
group) - Spam retention time
- Sender address restriction based on login name
- Dynamic group members address list expansion
nethserver-mail-ipaccess¶
IMAP access for a specific group of users. See IP-based IMAP access restriction.
nethserver-mail-getmail¶
The package configures getmail using cron.
For each enabled account the system:
- generates a
.cfg
file inside the/var/lib/getmail
directory from the template/etc/e-smith/templates/getmailrc
- adds a line inside the
/etc/cron.d/getmail
, all getmail instances use a non-blocking flock - delivers the messages using dovecot-lda
All operations are logged in /var/log/maillog
.
If a virus is found inside a received mail, the message is dropped.
The evidence of log in /var/log/maillog
:
Feb 14 18:19:10 vm5 clamd[1791]: instream(local): Eicar-Test-Signature FOUND
Feb 14 18:19:11 test getmail: msg 4/12 (702 bytes) msgid 000008bb5785fb1d from <root@test.neth.eu> dropped by filter Filter_classifier clamdscan (allow_root_commands="True", arguments="('-c', '/etc/clamd.d/amavisd.conf', '--stdout', '--no-summary', '--infected', '-')", command="clamdscan", exitcodes_drop="('1',)", exitcodes_keep="('0',)", group="None", ignore_stderr="False", path="/usr/bin/clamdscan", unixfrom="False", user="None")
nethserver-mail-p3scan¶
This package configures p3scan, full-transparent POP3 proxy-server for email clients.
- POP3 and POP3s proxy
- Anti-virus and anti-spam checks
mail-quarantine¶
This package makes a quarantine for spam. They are sent to a mailbox (you need to manually created it), waiting a review of the sysadmin. If enabled a mail notification is sent to the postmaster (root alias) for each quarantined email.
Database format¶
configuration¶
Postfix example:
postfix=service
...
AccessPolicies=
AlwaysBccStatus=disabled
AlwaysBccAddress=
MessageQueueLifetime=4
MessageSizeMax=20000000
ConnectionsLimit=
ConnectionsLimitPerIp=
SystemUserRecipientStatus=disabled
...
SenderValidation=disabled
DynamicGroupAlias=disabled
HeloHost=
SmartHostAuth=disabled
SmartHostAuthStatus=disabled
SmartHostName=192.168.5.252
SmartHostPassword=password
SmartHostPort=25
SmartHostStatus=disabled
SmartHostTlsStatus=enabled
SmartHostUsername=ns1
AccessPolicies
: A comma separated list of values. ObsoletesSubmissionPolicyType
prop. Currently defined values aresmtpauth
andtrustednetworks
.- smtpauth enable SMTP/AUTH on port 25, otherwise it is enabled only on 587 and 465 submission ports
- trustednetworks allow relay from any host in trusted networks (green network included).
AlwaysBccStatus {enabled,disabled}
: ifenabled
any message entering Postifx mail system is copied to the givenAlwaysBccAddress
.AlwaysBccAddress
: an email address that always receives a message copy (controlled byAlwaysBccStatus
).SystemUserRecipientStatus {enabled,disabled}
enabled
, accept from any network the recipient addresses formed by user account names and domain partlocalhost
,localhost.<domainname>
and FQDN hostname.SenderValidation {enabled,disabled}
, defaultdisabled
, checks the SMTP sender is consistent with /etc/login_maps and /etc/login_maps.pcre contents.DynamicGroupAlias {enabled,disabled}
, defaultdisabled
,- if
enabled
, create distribution lists based on system groups. See also the “Dynamic group aliases” section below.
HeloHost
. FQDN hostname used by Postfix when connecting to other MTAs
Dovecot example:
dovecot=service
...
AdminIsMaster=disabled
DeletedToTrash=disabled
FtsLuceneStatus=enabled
ImapStatus=enabled
LmtpInetListenerStatus=disabled
LogActions=disabled
MaxProcesses=400
MaxUserConnectionsPerIp=12
PopStatus=enabled
QuotaDefaultSize=20
QuotaStatus=disabled
SharedMailboxesStatus=enabled
SharedSeen=disabled
SpamFolder=Junk
SpamRetentionTime=15d
TlsSecurity=required
RestrictedAccessGroup=
Properties:
AdminIsMaster {enabled,disabled}
allow root user to impersonate other usersDeletedToTrash {enabled,disabled}
deletedtotrash pluginFtsLuceneStatus {enabled,disabled}
lucene indexed search pluginImapStatus {enabled,disabled}
IMAP protocol switchLmtpInetListenerStatus {enabled,disabled}
open a TCP socket for LMTP protocolLogActions {enabled,disabled}
IMAP actions logging pluginMaxProcesses N
maximum number of worker processes spawned by dovecot. A single user session usually requires multiple processes.MaxUserConnectionsPerIp N
maximum TCP connections for one user behind the same IP. This number will be multplied by 5 for connections coming from localhost.PopStatus {enabled,disabled}
POP3 protocol switchQuotaDefaultSize N
Default user quota size (1 unit is 10MB)QuotaStatus {enabled,disabled}
General user mail quota switchSharedMailboxesStatus {disabled,enabled}
Control the “Shared” IMAP namespace for per-user folder sharingSharedSeen {disabled,enabled}
Control the Seen IMAP flag (enabled means all users will see an email as read as soon as the first user reads it)SpamFolder FolderName
Deliver spam tagged messages to the given folder (applied to all users)SpamRetentionTime Nd
Expunge messages in SpamFolder if older than the given time span. “d” is for days.TlsSecurity {optional,required}
controls dovecotdisable_plaintext_auth
parameter: if set torequired
clear-text authentication methods are disabled, whileoptional
enables them.RestrictedAccessGroup
The value is a long group name, likedomain admins@mydomain.tld
. Members of the given group can login to dovecot services only from trusted networks. Install theVszLimit
set thedefault_vsz_limit
dovecot option, values are expressed inM
nethserver-mail-server-ipaccess
package to enable this feature.
p3scan example:
p3scan=service
SSLScan=enabled
SpamScan=enabled
TCPPort=8110
Template=/etc/p3scan/p3scan-en.mail
VirusScan=enabled
access=
status=enabled
rspamd example:
rspamd=service
BlockAttachmentClassList=Exec
BlockAttachmentCustomList=doc,odt
BlockAttachmentCustomStatus=disabled
BlockAttachmentStatus=enabled
OletoolsStatus=enabled
Password=uO9QjlnRCDsT0ZCD
RecipientWhiteList=
SenderBlackList=
SenderWhiteList=
SpamCheckStatus=enabled
SpamDsnLevel=20
SpamGreyLevel=4
SpamKillLevel=15
SpamSubjectPrefixStatus=disabled
SpamSubjectPrefixString=***SPAM***
SpamTag2Level=6
SpamTagLevel=2
VirusAction=reject
VirusCheckStatus=enabled
VirusScanOnlyAttachment=false
VirusScanSize=20000000
VirusScanTimeout=90
VirusScanSoftReject=enabled
status=enabled
Properties:
BlockAttachmentClassList {Exec,Arch}
Reject the attachements matching the extension listBlockAttachmentCustomList List
Reject the attachements matching the custom extension listBlockAttachmentCustomStatus {enabled,disabled}
Enable the custom list of rejected extensionsOletoolsStatus {enabled,disabled}
Enable Oletools to reject suspicious microsoft office document macroPassword
Password to authenticate the user rspamd for the Rspamd UIRecipientWhiteList
Do not perform checks for the recipient list, always acceptSenderBlackList
Do not perform checks for the sender list, always rejectSenderWhiteList
Do not perform checks for the sender list, always acceptSpamCheckStatus {enabled,disabled}
Enable the SPAM filterSpamSubjectPrefixStatus {enabled,disabled}
Enable to rewrite the subject when a possible spam is detectedSpamSubjectPrefixString string
Rewrite the subject with the string when a possible spam is detectedVirusAction
Possible action when a virus is detected (reject is default, ‘rewrite_subject’ to tag as spam)VirusCheckStatus {enabled,disabled}
Enable the virus check with ClamavVirusScanOnlyAttachment {true,false}
If true only messages with non-image attachments will be checkedVirusScanSize
The messages > n bytes in size are not scanned (valuable for Clamav and Oletools)VirusScanTimeout
Time in seconds that clamav is allowed to scan the email before to hit a timeout (default 90)VirusScanSoftReject {enabled,disabled}
when clamav hit a timeout to scan an email, if enabled the email is soft rejected (try again)
domains¶
Record of type domain:
internal.tld=domain
...
TransportType=none
mycompany.com=domain
...
TransportType=Relay
RelayHost=10.1.1.4
RelayPort=25
DisclaimerStatus=disabled
test.tld=domain
...
TransportType=SmtpSink
example.com=domain
...
TransportType=LocalDelivery
UnknownRecipientsActionType=deliver
UnknownRecipientsActionDeliverMailbox=jdoe
AlwaysBccStatus=enabled
AlwaysBccAddress=admin``there.org
other.net=domain
...
TransportType=Relay
RelayHost=mail.other.net
RelayPort=25
accounts¶
Groups:
employees@domain.com=group
...
MailStatus=enabled
MailAccess=private
administrators@domain.com=group
...
MailStatus=enabled
MailAccess=public
info@domain.com=group
...
MailStatus=enabled
MailAccess=public
User:
jdoe=user
FirstName=John
LastName=Doe
...
MailStatus=enabled
MailQuotaType=custom
MailQuotaCustom=15
MailForwardStatus=disabled
MailForwardAddress=
MailForwardKeepMessageCopy=no
and his pseudonyms: ::
john.doe``example.com=pseudonym
Account=jdoe
ControlledBy=system
Access=public
doe``=pseudonym
Account=jdoe
ControlledBy=operators
Access=private
getmail¶
All records of type getmail
are saved inside the getmail
database.
Properties:
- The key is the mail account to be downloaded
status
: can beenabled
ordisabled
, default isenabled
Account
: local user where messages will be delivered. Should be in the form user@domainServer
: server of the mail accountUsername
: user name of the mail accountPassword
: password of the mail accountDelete
: numbers of days after downloaded messages will be deleted,-1
means never,0
means immediatelyTime
: integer number rappresenting the minutes between each check, valid valued are between 1 and 60FilterCheck
: ifenabled
, check downloaded messages for viruses and spam usingrspamc
classifierRetriever
: can be any getmail retriever, see Getmail official doc- Retrievers available in the web interface:
SimplePOP3Retriever
SimplePOP3SSLRetriever
SimpleIMAPRetriever
SimpleIMAPSSLRetriever
Example:
db getmail set test@neth.eu getmail Account pippo@neth.eu status enabled Password Nethesis,1234 Server localhost Username test@neth.eu Retriever SimplePOP3Retriever Delete enabled Time 30 VirusCheck enabled SpamCheck enabled
quarantine¶
The properties are under the rspamd
key (configuration database):
rspamd=service
...
QuarantineAccount=vmail+quarantine
QuarantineSelector=is_reject
QuarantineStatus=enabled
SpamNotificationStatus=disabled
* ``QuarantineAccount``: The local email box where to send all spams (spam check is automatically disabled on this account). You must create it manually. You could send it to an external mailbox but then you must disable the spam check on this server
* ``QuarantineSelector``: It is possible to move to quarantine all spams (add_header, rewrite_subject, reject), allowed values are ``is_reject`` (default) or ``is_spam``
* ``QuarantineStatus``: Enable the quarantine, spam are no more rejected: enabled/disabled (default)
* ``SpamNotificationStatus``: Enable the email notification when email are quarantined: enabled/disabled (default)
For example, the following commands enable the quarantine:
config setprop rspamd QuarantineAccount spam@domain.org QuarantineStatus enabled SpamNotificationStatus enabled
signal-event nethserver-mail-quarantine-save
Mail quota¶
The default mail quota is configured in dovecot.conf
. Custom user mail quota
is set by the dovecot-postlogin
script, by reading
/etc/dovecot/user-quota
(which is a template). If a custom mail quota is set
the UI interface does not show the updated value until the user performs an IMAP
login.
Disabled users¶
By default all system users are also Dovecot users. To disable a user we
configure a blacklist in dovecot.conf
: /etc/dovecot/deny.passwd
.
As Dovecot is configured as authentication backend for Postfix, a disabled user loses also SMTP AUTH access.
Testing Dovecot with Mutt¶
Read admin’s mail with Mutt IMAP client. Quickstart:
yum install mutt
cat - <<EOF > ~/.muttrc
set spoolfile="imaps://root@localhost/"
set folder=""
EOF
mutt
See: http://dev.mutt.org/doc/manual.html
When mutt starts always asks for the root
password.
To avoid typing the password again and again write it in .muttrc
:
set spoolfile="imaps://root:PASSWORD@localhost/"
set folder=""
PASSWORD
must be URL-encoded. For instance the slash character /
is encoded as %2f
.
Set special ACL on mailboxes¶
The nethserver-mail-shrmbx-modify
action applies some predefined ACL
settings to shared mailboxes (type the mailbox name twice: the action performs also rename):
/etc/e-smith/events/actions/nethserver-mail-shrmbx-modify EVENT OLDNAME NEWNAME ID PERM [ID PERM ...]
For instance, let’s grant full “admin” permissions to group “administrators”:
/etc/e-smith/events/actions/nethserver-mail-shrmbx-modify ev 'Public folder1' 'Public Folder One' group=administrators@$(hostname -d) ADMIN
You can also use doveadm
to set special ACL on a shared mailbox:
doveadm acl set -u <user> <shared_mailbox> <subject> <flags>
Example: allow insert and expunge to user goofy on public mailbox testshare (domain of the machine is local.nethserver.org):
doveadm acl set -u goofy@local.nethserver.org Public/testshare "user=goofy@local.nethserver.org" insert expunge
IP-based IMAP access restriction¶
This feature allows to restrict IMAP access for a specific group. Members of the given group have IMAP access restricted to trusted networks.
Install
nethserver-mail-ipaccess
packageyum install nethserver-mail-ipaccess
Set the limited group, remember to use the full group name:
<group>@<domain>
config setprop dovecot RestrictedAccessGroup <group>@<domain>
Example for group
collaborators@nethserver.org
:config setprop dovecot RestrictedAccessGroup collaborators@nethserver.org
Apply the configuration
signal-event nethserver-mail-server-save
Syntax of /etc/dovecot/ipaccess.conf
¶
The dovecot-postlogin
script enforces an IP-based access policy to dovecot
services when the file :file:/etc/dovecot/ipaccess.conf
exists and is readable.
The file is composed by comments and records. Comments are line starting with #
,
whilst records have the following syntax:
<long group name> = <cidr list>
A long group name is the group name with domain suffix, like domain
admins@mydomain.tld
.
The cidr list is a comma-separated list of IP and network addresses in CIDR
format, like 127.0.0.1, 192.168.1.0/24, 10.1.1.2
. The binary conversion is
implemented by the NetAddr::IP
Perl module. See perldoc NetAddr::IP
for
details.
Enable dovecot IMAP rawlog¶
This section describes how to record the list of IMAP commands sent by the client and the server during an IMAP session. For more information see Dovecot rawlog.
Create the file
/etc/e-smith/templates-custom/etc/dovecot/dovecot.conf/90rawlog
with the
following contents:
#
# 90rawlog (custom)
#
import_environment = $import_environment DEBUG=1
service imap-postlogin \{
executable = script-login -d rawlog -t /usr/libexec/nethserver/dovecot-postlogin
\}
Apply the new configuration
signal-event nethserver-mail-server-save
To enable IMAP rawlog for a specific user, identify the user (vmail) home directory with the following command:
# doveadm user -u first.user@dpnet.nethesis.it
userdb: first.user@dpnet.nethesis.it
system_groups_user: first.user@dpnet.nethesis.it
uid : 987
gid : 990
home : /var/lib/nethserver/vmail/first.user@dpnet.nethesis.it
Warning
Always use the user long name form, which includes the @domain
suffix.
In our example first.user
would not work
Create the dovecot.rawlog
directory and change permissions:
mkdir "/var/lib/nethserver/vmail/first.user@dpnet.nethesis.it/dovecot.rawlog"
chown vmail:vmail "/var/lib/nethserver/vmail/first.user@dpnet.nethesis.it/dovecot.rawlog"
Detailed IMAP rawlogs are now created under the user’s (vmail) home directory.
Each session is split into two files: .in
file for client requests, .out
file for server responses. For instance,
/var/lib/nethserver/vmail/first.user@dpnet.nethesis.it/dovecot.rawlog/20180913-143301-6293.in
/var/lib/nethserver/vmail/first.user@dpnet.nethesis.it/dovecot.rawlog/20180913-143301-6293.out
The initial timestamp helps to mix them together and obtain the complete IMAP session trace:
sort -n /var/lib/nethserver/vmail/first.user@dpnet.nethesis.it/dovecot.rawlog/20180913-143301-6293.*
Access the rspamd UI¶
The rspamd UI is available from the same httpd instance of Server Manager:
https://<IP>:980/rspamd
Access is granted to any account defined in
/etc/httpd/admin-conf/rspamd.secret
. By default a rspamd
login is
created automatically. Its password is available with the following command:
config getprop rspamd Password
Additional accounts can be created with the following command:
/usr/bin/htpasswd -b -m /etc/httpd/admin-conf/rspamd.secret username S3cr3t
If an account provider is configured, the default access policy to rspamd UI is
granting access also to admin
user and members of the domain admins
group.
Type config show admins
for details.
Bayesian rules upgrade to rspamd¶
Each Junk
(or junkmail
) folder from users’ accounts, if present, can be
used to train the Rspamd Bayesian filter database, by running the attached
script:
bash /usr/share/doc/nethserver-mail-server-*/bayes_training.sh
Sender address validation¶
If the postfix/SenderValidation
prop is set to enabled
the SMTP server
restricts the Mail from
command usage. The sender address must be associated
with the SMTP login name. The login/sender match is specified in the following
Postfix tables, both implemented with an e-smith template:
/etc/postfix/login_maps
/etc/postfix/login_maps.pcre
To enable the SenderValidation
:
config setprop postfix SenderValidation enabled
signal-event nethserver-mail-server-update
Postfix SMTP listening ports¶
The Postfix SMTP server listens on the following TCP ports
25
, standard SMTP port; used by other MTAs587
, standard SMTP submission port; STARTTLS required by default to protect login passwords; used by MUAs465
, standard SMTPS submission port; TLS always required at socket level; used by MUAs which not support STARTTLS10587
, additional SMTP submission port for localhost only; no TLS required; used by local mail applications
Dynamic group aliases¶
If the postfix/DynamicGroupAlias
prop is enabled
an additional
virtual_alias_maps
TCP table is available. It expands a long group name to
the group members list with a getent group
call. The table is implemented in
:file:/usr/libexec/nethserver/postfix-get-group
. Note that group members
lists are returned by SSSD: as such they obey to its caching rules.
When the DynamicGroupAlias
general switch is enabled, individual groups can
be disabled and marked private. If MailStatus
prop is disabled
the
group long name is not considered a valid email address anymore. The
MailAccess
prop works like the Access
prop for user
records: if set
to private
only authenticated SMTP clients are allowed to use it as
recipient.
Accounts DB group
props example:
employees@domain.com=group
...
MailStatus=enabled
MailAccess=private
nethserver-makerpms¶
RPM builds by Linux containers
This is a simple RPM build environment based on the official CentOS Docker image.
It can build RPMs in the github actions environment, or on your local Fedora 31+/CentOS 8 machine.
Installation/upgrade¶
On Fedora 31+ and CentOS 8, run as a non-root user
$ curl https://raw.githubusercontent.com/NethServer/nethserver-makerpms/master/install.sh | bash
There must be podman
[1] already installed though.
Build requirements¶
Requirements for building RPMs, implemented by the buildimage/makesrpm
command:
- The .spec file is in the current working directory
- Source tarballs defined in the .spec file can be downloaded via
http://
orftp://
. - If a
SHA1SUM
file exists in the current working directory, the integrity of source tarballs is checked against it. Do not put any path in theSHA1SUM
contents, only the tarball/source file name. - If the current working directory contains a
.git
directory (thus it is a git repository workdir) the specfilesource0
can be set toSource: %{name}-%{version}.tar.gz
. Then git-archive generates a tarball of the repository itself at the HEAD commit. - Other tarballs are downloaded automatically with
spectool
during the build.
Building RPMs locally¶
If the previous build requirements are met, change directory to the repository root then run
$ makerpms *.spec
Run without any argument to get a brief help
$ makerpms
To build a package for another distribution pass the DIST
environment variable
$ DIST=.el7 makerpms *.spec
If you have a custom or development builder image to test, set the IMAGE
environment variable, e.g.:
$ IMAGE=me/myimage:test makerpms *.spec
It is possible to override the builder command, with the COMMAND
environment variable
$ COMMAND="whoami" makerpms *.spec
makerpms
Additional arguments can be passed to YUM before starting the binary build, to fetch dependencies
not tracked by BuildRequires
, enable additional repositories and so on…
$ YUM_ARGS="--enablerepo=nethserver-testing" makerpms *.spec
To pass additional arguments to Podman, use PODMAN_ARGS instead
$ PODMAN_ARGS="--volume=$PWD/myrepo.conf:/etc/yum.repos.d/myrepo.conf"
Optimizations¶
To speed up the build process, the YUM cache directory contents are preserved.
Container instances share the named Podman volume makerpms-yum-cache
.
To clear the YUM cache run
$ podman volume rm makerpms-yum-cache
Builder container images¶
The builder container images are updated periodically and are available at https://hub.docker.com/r/nethserver/makerpms.
nethserver/makerpms:7
is the default image, fornoarch
buildsnethserver/makerpms:buildsys7
is the image forx86_64
builds (GCC 4)nethserver/makerpms:devtoolset7
is the image forx86_64
builds with GCC 9 (devtoolset-9 from SCLo), then run makerpms in a SCLo environment, e.g. :$ COMMAND="scl enable devtoolset-9 -- makerpms" makerpms *.spec
For more info about how to build the images locally look at travis/build-container.sh
.
The following command locally builds the images with Podman (instead of Docker):
$ cat travis/build-container.sh | sed s/docker/podman/ | \
grep -v -E '\b(login|logout|push)\b' | \
NSVER=7 IMAGE_REPO=me/myimage bash
To test the local image run makerpms as follow:
$ IMAGE=me/myimage:7 makerpms *.spec
Images for NethServer 6 are available as well: just replace 7
with 6
.
Other commands¶
The install.sh
scripts installs also the following commands:
uploadrpms
is a RPM publishing helper, specific for the NethServer community RPMs publishing policiesreleasetag
is a release workflow helper, specific for the NethServer community release guidelinesmakesrpm
builds just the.src.rpm
package.
uploadrpms¶
The first time, before running uploadrpms
ensure the following command works
$ sftp username@packages.nethserver.org
Accept the server SSH key fingerprint when asked.
The following command uploads all the RPMs in the current working directory to the nethforge
testing
repository for NethServer version 7.8.2003
.
$ uploadrpms username@packages.nethserver.org:nscom/7.8.2003/nethforge-testing *.rpm
Replace 7.8.2003
with the correct NS version number. Also replace nethforge-testing
with the target repository name.
The command output might complain about some SFTP disabled commands. Ignore those messages.
releasetag¶
The releasetag
command executes a workflow that suits only those
NethServer packages that expect a Version
tag in the form X.Y.Z
.
Some RPMs, (notably nethserver-release
) require a different version schema
and releasetag
does not suit their release workflow. Refer to their README
files for more information.
When releasetag
is invoked:
- reads the git log history and fetches related issues from the issue tracker web site.
- updates the
Version
tag and the%changelog
section in the.spec
file. - commits changes to the
.spec
file. - tags the commit (with optional GPG signature).
To fetch issues from private GitHub repositories
create a private GitHub access token.
Select the repo
scope only.
Copy it to ~/.release_tag_token
and keep its content secret:
chmod 600 ~/.release_tag_token
Tip
The private access token is useful also for public repositories because authenticated requests have an higher API rate limit
The releasetag
command is now ready for use. This is the help output:
releasetag -h
Usage: releasetag [-h] [-k KEYID] [-T <x.y.z>] [<file>.spec]
Sample invocation:
releasetag -k ABCDABCD -T 1.8.5 nethserver-mail-server.spec
To force a local GPG password prompt (tested on Fedora) prepend some additional environment variables:
GPG_TTY=$(tty) GPG_AGENT_INFO="" releasetag -k ABCDABCD -T 1.8.5 nethserver-mail-server.spec
Replace ABCDABCD
with your signing GPG key. The $EDITOR
program (or git core.editor
) is opened automatically to adjust the
commit message. The same text is used as tag annotation.
Usage of -k
option is optional.
To push the tagged release to GitHub (and possibly trigger an automated build)
ensure to add the --follow-tags
option to git push
invocation. For
instance:
git push --follow-tags
To make --follow-tags
permanent run this command:
git config --global push.followTags true
Building RPMs on github.com¶
github.com automatically builds RPMs and uploads
them to packages.nethserver.org
, if configured with enough environment variables
and upload secrets.
Configuration¶
To automate the RPM build process using GitHub
- create a
.github/workflows/make-rpms.yml
file inside the source code repository hosted on GitHub. - the repository must have actions enabled and upload secrets properly set up, please contact the organization maintainer on community.nethserver.org for help.
This is an example of .github/workflows/make-rpms.yml
contents:
name: Make RPMs
on:
push:
branches:
- master
- main
pull_request:
jobs:
make-rpm:
runs-on: ubuntu-22.04
env:
dest_id: core
docker_image: ghcr.io/nethserver/makerpms:7
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
ref: ${{ github.head_ref }}
- name: Generate .env file
run: |
cat > .env <<EOF
DEST_ID=${{ env.dest_id }}
NSVER=7
DOCKER_IMAGE=${{ env.docker_image }}
GITHUB_ACTIONS=1
GITHUB_HEAD_REF=${{ github.head_ref }}
GITHUB_REF=${{ github.ref }}
GITHUB_REPOSITORY=${{ github.repository }}
GITHUB_RUN_ID=${{ github.run_id }}
ENDPOINTS_PACK=${{ secrets.endpoints_pack }}
SECRET=${{ secrets.secret }}
SECRET_URL=${{ secrets.secret_url }}
AUTOBUILD_SECRET=${{ secrets.autobuild_secret }}
AUTOBUILD_SECRET_URL=${{ secrets.autobuild_secret_url }}
EOF
- name: Run prep-sources if present.
run: if test -f "prep-sources"; then ./prep-sources; fi
- name: Build RPM and publish
run: |
echo "Starting build..."
docker run --name makerpms \
--env-file .env \
--hostname $GITHUB_RUN_ID-$GITHUB_RUN_NUMBER.nethserver.org \
--volume $PWD:/srv/makerpms/src:ro \
${{ env.docker_image }} \
makerpms-github -s *.spec
echo "Build succesful."
if [[ "${{ secrets.endpoints_pack }}" && "${{ secrets.secret }}" ]]; then
echo "Publish configuration exists, pushing package to repo."
docker commit makerpms nethserver/build
docker run \
--env-file .env \
nethserver/build \
uploadrpms-github
echo "Publish complete."
fi
rm .env
Usage¶
GitHub Actions builds are triggered automatically when:
- one or more commits are pushed to the master or main branch of the NethServer repository, as
stated in the
.github/workflows/make-rpms.yml
file inside theon.push.branches
key - A pull request is opened from a NethServer repository fork or it is updated by submitting new commits
After a successful build, (if secrets are available) the RPM is uploaded to packages.nethserver.org
,
according to the DEST_ID
variable value. Supported values are core
for
NethServer core packages, and forge
for NethForge packages.
Pull requests are commented automatically by nethbot
[4] with the links to available RPMs.
Also issues are commented by nethbot
if the following rules are respected in git commit messages:
- The issue reference (e.g.
NethServer/dev#1234
) is present in the merge commit of pull requests - The issue reference is added to standalone commits (should be rarely used)
The build environment supports the following variables:
DOCKER_IMAGE
DEST_ID
DOCKER_IMAGE¶
The Docker build image can contain different RPMs depending on the tag:
latest
(or7
) contains only dependencies to buildnethserver-*
RPMS, likenethserver-base
. It actually installs only nethserver-devtools and a basic RPM build environment without gcc compiler.buildsys7
is based on the previous environment. It also pulls in the dependencies for arch-dependant packages (likeasterisk13
orns-samba
). It actually installs thebuildsys-build
package group, which provides thegcc
compiler (version 4) among other packages.devtoolset7
: it extends the buildsys7 with the devtoolset-9 SCLo packages set. It is possible to compile with gcc version 9, by prefixing the container entry point in the following way:docker run -ti [OPTIONS] scl enable devtoolset-9 -- makerpms-travis package.spec
For instance, see the sofia-sip package.
DEST_ID¶
If DEST_ID=core
:
- Builds triggered by pull requests are uploaded to the
autobuild
[2] repository - Builds triggered by commits pushed to master are uploaded to the
testing
[3] repository. If a git tag is on the last available commit, the upload destination is theupdates
repository.
If DEST_ID=forge
:
- Pull requests are uploaded to
nethforge-autobuild
- Branch builds are uploaded to
nethforge-testing
, whilst tagged builds are uploaded tonethforge
Warning
In any case, the git tag must begin with a digit
and not containing any “-” minus symbol.
For instance the tag 0.1.12
is considered
as a tagged build whilst v0.1.12-1
is not
References
[1] | Podman is a daemonless Linux container engine. https://podman.io/ |
[2] | Is a particular kind of repository in packages.nethserver.org that hosts the rpms builded automatically from github.com/features/actions. http://packages.nethserver.org/nethserver/7.9.2009/autobuild/x86_64/Packages/ |
[3] | Is a repository in packages.nethserver.org that hosts the rpms builded automatically from github.com/features/actions started form official nethserver github repository. http://packages.nethserver.org/nethserver/7.9.2009/testing/x86_64/Packages/ |
[4] | Is our bot that comments the issues and pull request with the list of automated RPMs builds. https://github.com/nethbot |
nethserver-mattermost¶
Stack:
- Mattermost
- PostgreSQL 12 listening on non-standard port 55434
- Apache as proxy server
Apache configuration derived from: https://github.com/mattermost/docs/issues/1114
First configuration¶
Mattermost requires a dedicated virtualhost and it’s accessibile only from HTTPS.
To start Mattermost, execute:
config setprop mattermost VirtualHost mattermost.yourdomain.com status enabled
signal-event nethserver-mattermost-update
Then, access https://mattermost.yourdomain.com
and perform the first configuration.
Database¶
Properties:
TCPPort
: Mattermost listen port, change only for development purposeVirtualHost
: dedicated FQDN for virtual hoststatus
: can beenabled
ordisabled
, default todisabled
access
: firewall access, leave blank or at least set tonone
Example:
mattermost=service
TCPPort=5432
VirtualHost=mattermost.local.neth.eu
access=
status=enabled
Account synchronization¶
The mattermost-bulk-user-create
command will:
- create a default team named as the
Company
fromOrganizationContact
- read all users from local or remote Account Providers and create them inside Mattermost
Please note that:
- users disabled in the Server Manager or already existing in Mattermost will be skipped
- a random password will be generated for each user
Forcing a common default password¶
It’s possible to set a default password for each new Mattermost user, just append the default password to command invocation.
Example:
mattermost-bulk-user-create Password,1234
Log files¶
The mattermost
service logs are created under the /opt/mattermost/logs/
directory
and are rotated by Mattermost itself.
Cockpit API¶
read¶
Return mattermost status and configuration. No input required.
Output¶
Example:
{
"status": {
"users": "2",
"teams": "1",
"channels": "2",
"posts": "2"
},
"configuration": {
"props": {
"status": "enabled",
"access": "",
"VirtualHost": "mattermost.nethserver.org",
"TCPPort": "5432"
},
"name": "mattermost",
"type": "service"
}
}
validate¶
Constraints:
VirtualHost
: must be a valid FQDN
Input¶
Example:
{
"props": {
"status": "enabled",
"VirtualHost": "mattermost.nethserver.org",
},
"name": "mattermost",
}
update¶
Same input as validate.
nethserver-mock¶
The nethserver-mock
package provides some scripts to ease the process of
building and releasing RPMs.
Configuring the environment¶
On NethServer, install nethserver-mock
package, by typing:
yum install nethserver-mock
On Fedora, and other RPM-based distros run the command:
yum install <URL>
Or
dnf install <URL>
where <URL> is http://packages.nethserver.org/nethserver/7.3.1611/base/x86_64/Packages/nethserver-mock-1.3.2-1.ns7.noarch.rpm at the time of writing.
The build process uses Mock and must be run as a non privileged user,
member of the mock
system group. Add your user to the mock
group:
usermod -a -G mock <username>
Running the scripts¶
The make-rpms
command eases building of the NethServer RPMs by
hiding the complexity of other commands. It is designed to work
inside the git repository directory of NethServer packages, but should
fit other environments, too.
Start by cloning the git repository and move inside it. For instance
git clone https://github.com/nethesis/nethserver-mail-server.git
cd nethserver-mail-server
To build the RPM just type
make-rpms nethserver-mail-server.spec
The command writes the results into the current directory, assuming every change to the source code has been commited. If everything goes well they consist of:
- source RPM
- binary/noarch RPMs
- mock log files
To clean up the git repository directory, git clean
may help:
git clean -x -n
Substitute -n
with -f
to actually remove the files!
Note
The make-rpms
command is sensible to dist
and mockcfg
environment variables. If they are missing the default values are
shown by invoking it without arguments.
For example:
dist=ns7 mockcfg=nethserver-7-x86_64 make-rpms *.spec
The make-rpms
command in turn relies on other scripts
make-srpm
- Builds the
.src.rpm
file. prep-sources
- Extracts and/or fetches the source tarballs.
The first Source
tag in the .spec
file is assumed refer to
the local git repository. If an absolute URL is specified, only the
last part is considered. Other SourceN
tags must conform to the
Fedora RPM guidelines [1]. The external sources are actually
fetched by the spectool
command.
If the file SHA1SUM
exists in the same directory of the
.spec
file the tarballs are checked against it.
Development and Release builds¶
During the development, a package can be rebuilt frequently: increasing build numbers and unique release identifiers are useful during this stage to help the whole process.
When make-rpms
is invoked, it checks the git log history and tags
to decide what kind of build is required: development or release.
Release builds produce a traditional RPM file name, i.e.:
nethserver-mail-server-1.8.4-1.ns6.noarch.rpm
Development builds produces a marked RPM, i.e:
nethserver-mail-server-1.8.3-1.6gite86697e.ns6.noarch.rpm
Other differences in development from release are
- the
%changelog
section in.spec
is replaced by the git log history since the last tag - the number of commits since the last tag, and the latest git commit
hash are extracted from
git describe
and prepended to the%dist
macro.
Signing RPMs¶
The command sign-rpms
is a wrapper around rpm --resign
command. Its advantage is it can read a password for the GPG
signature from the filesystem. Sample invocation:
sign-rpms -f ~/.secret -k ABCDABCD
The signature is added automatically by packages.nethserver.org
.
Running on EL8¶
It’s possible to use nethserver-mock
also on EL8 (CentOS 8).
If the build fails, please remove dnf-plugin-spacewalk
RPM from the system.
References
[1] | Referencing Source http://fedoraproject.org/wiki/Packaging:SourceURL |
nethserver-mysql¶
Manage and configure MySQL server.
When installed, the module will trigger the mysql_secure_installation
script generating default configuration:
- Auto-generated root password saved in /var/lib/nethserver/secrets/mysql
- No anonymous user
- No remote root login allowed
- No test database
Differences from SME Server:
- InnoDB tables enabled, removed
InnoDB
property - TCP connections enabled: can be disabled setting
LocalNetworkingOnly
property toyes
- Removed
OpenFilesLimit
property max_allowed_packet
option is 1MB (as MySQL default). To change the value set theMaxAllowedPacket
property
The root password is also saved in /root/.my.cnf
, so the root user can use mysql client without typing any password.
Database example:
mysqld=service
LocalNetworkingOnly=no
MaxAllowedPacket=16MB
TCPPort=3306
status=enabled
Loading data¶
There is a special action /etc/e-smith/events/actions/nethserver-mysql-init
which can be used to load data inside the database.
OBSOLETE: For backward compatibility with SME Server, the mysql.init
service can also be used instead of the new action. The use of the service is HIGHLY discouraged.
When launched, the script will search for links in /etc/e-smith/sql/init
directory.
If a link point to a file with .sql extension, the content of the file will be loaded using the mysql
command.
Otherwise if the file is executable it will be simply executed.
If no error occurs, the link will be removed.
Given ad simple /tmp/test.sql
script, you can load it using this commands:
ln -s /tmp/test.sql /etc/e-smith/sql/init/temp.sql
service mysql.init start
nethserver-net-snmp¶
Simple SNMP server.
Package: nethserver-net-snmp.
Database¶
Configuration is saved in snmpd
key inside configuration
database.
Available properties:
Community
: name of the SNMP community, default ispublic
SysLocation
: name of the server location, default isUnknown
SysContact
: name and mail address of the system administrator, default isRoot <root@localhost>
.
Example:
snmpd=service
Community=public
SysLocation=Unknown
SysContact=Root <root@localhost>
status=enabled
access=private
UDPPort=61
Query example:
snmpwalk -mALL -v2c -cpublic localhost
Configuration can be applied using the nethserver-net-snmp-save
event.
nethserver-netdata¶
Basic configuration for netdata.
nethserver-cockpit package access netadata on http://localhost:19999
and uses the data to graph system statistics.
Do not change listening port otherwise Cockpit will not be able to access netdata.
Configuration DB¶
Properties:
Bind
: bind IP addess, default tolocalhost
Alarms
: can beenabled
ordisabled
. Default todisabled
to avoid unwanted mail to root userPlugins
: a comma-separated list of key and value for standard plugins. Example:tc:yes,charts.d:yes
PythonPlugins
: a comma-separated list of key and value for python plugins. Example:apache:yes,sensors:no
Please be careful when changing Plugins
and PythonPlugins
properties, because some plugins are used by Cockpit
to create charts inside multiple pages.
Database example:
netdata=service
Alarms=disabled
Bind=localhost
TCPPort=19999
access=
Plugins=proc:yes,tc:yes,charts.d:yes,python.d:yes,idlejitter:no,cgroups:no,checks:no,apps:no,node.d:no
PythonPlugins=apache:yes,sensors:yes
status=enabled
Access from LAN¶
To enable the access from LAN, execute the following:
config setprop netdata Bind 0.0.0.0 access green
signal-event nethserver-netdata-update
signal-event firewall-adjust
Access netdata from http://<server_green_ip>:19999
.
Links¶
- Offical netdata site: https://my-netdata.io/
nethserver-nextcloud¶
This package can be installed before or after any user provider like nethserver-dc and nethserver-directory.
If nethserver-dc or nethserver-directory are installed, the nethserver-nextcloud-save event will automatically enable all local users.
The package does the following:
- create
nextcloud
mysql database - create default database credentials: user nextcloud and password stored in
/var/lib/nethserver/secrets/nextcloud
- add trusted domains to use with web access
- create default credentials for web login: user “admin” and password “Nethesis,1234”
- set english as the default language
- set the user data directory as
/var/lib/nethserver/nextcloud
The configuration is stored inside the configuration
db, under the nextcloud
key. To show it:
config show nextcloud
Properties:
TrustedDomains
: list of trusted domains added to Nextcloud config fileVirtualHost
: set custom virtual host, e.g. mycloud.mydomain.itWellknown
: can beenabled
ordisabled
. If enabled, add redirects for calDAV and cardDAV. This property has effect only ifVirtualHost
is empty.HonorAdStartTls
: can beenabled
ordisabled
. The StartTLS option from SSSD configuration was historyically ignored. If set toenabled
, thesssd/StartTls
prop value is honored.
Admin user¶
After installation the application is accesible using the following credentials:
- User: admin
- Password: Nethesis,1234
Please, remember to change the default password after the first login!
Customize virtual host¶
Set custom virtual host and add it to trusted domains as follow:
config setprop nextcloud VirtualHost <VHOST>.<DOMAIN_NAME>
config setprop nextcloud TrustedDomains <VHOST>.<DOMAIN_NAME>
signal-event nethserver-nextcloud-update
Backup¶
The Nextcloud backup includes the configuration file and all data of the users:
/var/lib/nethserver/nextcloud
/usr/share/nextcloud/config/config.php
The database is automatically saved by nethserver-mysql
.
OCC¶
When using occ
command, PHP 7.3 should be enabled inside the environment.
Invocation example:
occ ldap:show-config"
The occ
command is just a wrapper around:
su - apache -s /bin/bash -c "source /opt/remi/php80/enable; cd /usr/share/nextcloud/; php occ ldap:show-config"
Log of php80-fpm¶
The log of php80-fpm can be found at /opt/remi/php80/log/php-fpm/error-nextcloud.log
Cockpit API¶
read¶
Return mattermost status and configuration.
Input¶
app-info
orconfiguration
Output¶
Example (app-info
):
{
"url": "https://your.host.domain"
}
Example (configuration
):
{
"props": {
"Wellknown": "enabled",
"VirtualHost": "eccoci.rva.org",
"TrustedDomains": "a.b,b.c"
},
"stats": {
"version": "16.0.2.1",
"admin_pass_warn": true,
"users": "3"
}
}
validate¶
Constraints:
VirtualHost
: must be a valid FQDN or EMPTYTrustedDomains
: must be an array of valid FQDN
Input¶
Example:
{
"props": {
"Wellknown": "enabled",
"VirtualHost": "eccoci.rva.org",
"TrustedDomains": "a.b,b.c"
}
}
update¶
Same input as validate.
Full reinstall¶
As with many other applications in NethServer, un-installing the Nextcloud application does not remove the settings, stored files, or the database. Here are the suggested steps to do a full un-install and re-install with a fresh configuration:
- Uninstall Nextcloud using the admin page
- Drop the MySQL database:
/opt/rh/rh-mariadb105/root/bin/mysql --socket="/var/run/rh-mariadb105-mariadb/nextcloud-mysql.sock" -e "drop database nextcloud;"
- Remove the packages:
yum remove nethserver-nextcloud
- Remove the whole Nextcloud directory:
rm -rf /usr/share/nextcloud/
- Remove the e-smith DB configuration:
config delete nextcloud
- Remove the NethServer config directory (WARNING: will remove user data):
rm -rf /var/lib/nethserver/nextcloud
- Install Nextcloud from the Software Center
nethserver-ns8-migration¶
This package will migrate services from NethServer 7 to NethServer 8 (NS8).
On install, the nethserver-ns8-migration-update
event creates a
WireGuard private key. This key is later used to join the NS8 cluster VPN.
The ns8-join
command adds NS7 to the cluster VPN and retrieves the its
configuration:
ns8-join [--no-tlsverify] <ns8_host> <admin_user> <admin_pass>
During ns8-join
execution, the nethserver-ns8-migration-save
event
establishes the VPN connection.
Once the VPN is established:
- subsequent NS8 API calls are routed through it
- a temporary NS8 external user domain is created
Warning
The NS8 cluster must be new, with no user domains and installed modules
Example:
ns8-join ns8.nethserver.org admin Nethesis,1234
After the join, it is possible to execute actions:
ns8-action <agent> <action> <json_data>
Example:
ns8-action cluster get-cluster-status '{}'
When NS7 modules are successfully migrated to NS8, run the ns8-leave
command.
ns8-leave
That command:
- ensures the temporary external user domain is removed
- removes the NS7 node from the VPN
- stops the Wireguard VPN and cleans up its settings
Commands¶
This is the full list of commands provided by the tool. They are used by applications and APIs:
ns8-join
ns8-leave
ns8-action
ns8-bind-app
ns8-abort
Applications¶
The migration tool can migrate a pre-defined set of NS7 applications (apps):
- nethserver-mail (with nethserver-webtop5, nethserver-roundcubemail)
- nethserver-nextcloud
- nethserver-mattermost
- nethserver-ejabberd
- account-provider (both local AD and LDAP)
Each application has two directories
- sources (or install) directory with the migration implementation, under
/usr/share/nethesis/nethserver-ns8-migration/apps/
. - state directory, with current migration state information, under
/var/lib/nethserver/nethserver-ns8-migration/
The app sources directory must provide the following files, and implement the described behavior:
bind.env
, basic environment variables for thebind
command.MODULE_IMAGE_URL
the Podman image URL to install and start on the remote moduleMODULE_VOLUMES
a list of name of Podman volumes that are created by the bind command on the remote side. These volumes can be filled by rsync uploads, together with the remote modulestate/
directory.
bind
command. This command creates a remote NS8 module instance and saves the Rsync configuration in the app state directory.migrate
command. This command synchronizes the local application configuration and data with the remote Rsync endpoint. If it is passed theMIGRATE_ACTION=finish
environment variable, it also finalizes the migration: it stops the local services and starts the remote ones.
Optionally the app can provide jq input translation filters for the migration APIs:
input-start.jq
input-finish.jq
The filter output is Bash-eval’ed and must define additional environment
variables for the app migrate
command.
Applications can read additional environment files, provided by the RPM package or created at runtime by the tool commands:
/etc/nethserver/agent.env
/var/lib/nethserver/nethserver-ns8-migration/agent.env
/var/lib/nethserver/nethserver-ns8-migration/environment
Nextcloud migration¶
Bind command example
MODULE_NODE_ID=1 ./bind
Sync command example
./migrate
Finish command example
MIGRATE_ACTION=finish NEXTCLOUD_VHOST=nc.example.com ./migrate
Ejabberd migration¶
Bind command example
MODULE_NODE_ID=1 ./bind
Sync command example
./migrate
Finish command example
MIGRATE_ACTION=finish ./migrate
Mattermost migration¶
Bind command example
MODULE_NODE_ID=1 ./bind
Sync command example
./migrate
Finish command example
MIGRATE_ACTION=finish MATTERMOST_VHOST=mattermost.example.com ./migrate
Email, Webtop, Roundcube migration¶
As both Webtop and Roundcube depends on the Email application, the migration of the three modules must occur at the same time and is controlled by the nethserver-mail app.
Bind command example
MODULE_NODE_ID=1 WEBTOP_NODE_ID=1 ROUNDCUBE_NODE_ID=2 ./bind
Sync command example
./migrate
Finish command example
MIGRATE_ACTION=finish WEBTOP_VHOST=webtop.example.com ROUNDCUBE_VHOST=rc.example.com ./migrate
Just for environment var reference, to finalize nethserver-webtop5 alone
MIGRATE_ACTION=finish MAIL_INSTANCE_ID=mail1 WEBTOP_VHOST=webtop.example.com ./migrate
Finally, to finalize nethserver-roundcubemail alone
MIGRATE_ACTION=finish MAIL_INSTANCE_ID=mail1 ROUNDCUBE_VHOST=rc.example.com ./migrate
Account provider¶
This application migrates the local account provider. Both AD and LDAP are handled. External account provider is not migrated: it must be manually configured in NS8 to reach the same LDAP server used by NS7.
File server¶
The Samba file server migration is part of Samba Account provider
migration. It occurs unless the skip
flag is set for the
nethserver-samba
application.
Migration APIs¶
The API responsible for apps migration is api/migration/update
. Its
basic input payload format is
{
"app": "nethserver-testapp",
"action": "start",
"migrationConfig": {
"appNode": 3
}
}
It accepts the following action
values for each NS7 module: start
,
sync
, finish
, abort
.
start
. Creates one module instance in the NS8 cluster. The local NS7 appbind
script is called. Multiple destination modules are allowed too: for instance the nethserver-mail app controls the migration of nethserver-webtop5 and nethserver-roundcubemail, if they are installed.sync
. Synchronizes local app configuration and data with the remote module instance, by calling itsmigrate
script.finish
. Completes the migration by calling the appmigrate
script with the special environment variableMIGRATE_ACTION=finish
.abort
. Abort module migration. Remove module from NS8 cluster and cleanup local stace.
After the execution of the finish
action the app is stopped and
disabled in NS7.
The API api/migration/read
returns the current migration status, for
each known app:
echo '{"action":"listApps"}' | /usr/libexec/nethserver/api/nethserver-ns8-migration/migration/read
Package uninstallation¶
To remove the tool and its dependencies (if they are not required by other packages):
yum remove nethserver-ns8-migration kmod-wireguard wireguard-tools
Clean up configuration database:
config delete wg-quick@wg0
config delete agent
config delete ns8
Post-migration step back¶
Once a service has been migrated to the remote NS8 host it should not run
any more on NS7. When the migrate
command completes the application
services are stopped and disabled.
Please note that some migrated applications may also add some custom templates. To list such template fragments use:
grep -lR ns8migration /etc/e-smith/templates-custom/
It is possible to manually re-enable the services with the following commands.
# Mail
config setprop dovecot status enabled
config setprop postfix status enabled
config setprop rspamd status enabled
config setprop opendkim status enabled
config setprop olefy status enabled
# Webtop
config setprop tomcat8@webtop status enabled
# Roundcube
config delprop roundcubemail migration
# Mattermost
config setprop mattermost status enabled
rm -rf /etc/e-smith/templates-custom/etc/httpd/conf.d/zz_mattermost.conf
signal-event nethserver-mattermost-update
# Nextcloud
rm -rf /etc/e-smith/templates-custom/etc/httpd/conf.d/zz_nextcloud.conf
rm -f /etc/e-smith/templates-custom/etc/httpd/conf.d/default-virtualhost.inc/40nextcloud
signal-event nethserver-nextcloud-update
# Account provider
config setprop slapd status enabled
config setprop nsdc status enabled
config setprop sssd status enabled
# File server
config setprop smb status enabled
config setprop smb nmb enabled
config setprop smb winbind enabled
# All modules
signal-event nethserver-ns8-migration-update
signal-event runlevel-adjust
signal-event firewall-adjust
Migration notes¶
Warning
Read carefully the sections below before finishing the migration of any application.
Webtop¶
If you purchased a Webtop license for additional custom fields or other components/integrations the following additional and manual steps are needed:
- Before finishing the Mail app migration, access the Webtop administrative page and disable any subscribed license.
- Finish the Mail app migration.
- In the NS8 module, access the administrative page and enable the licenses again.
File server¶
File server migration (shared folders) is an optional step of the Samba account provider migration. It can be performed if the NS8-based DC IP address is in a private network and is routable from the NS7-based DC.
The NS8 DC cannot be assigned the cluster VPN IP address.
Warning
Exposing SMB and other AD services to public networks is dangerous.
Account provider¶
When the migration finishes, the local account provider (both AD and LDAP) is stopped and disabled. In this state, SSSD allows logging on the system with its local cache. If some services are left on the system it is necessary to remove the local account provider and configure NS8 as the remote account provider.
nethserver-ntopng¶
After installation, ntopng is disabled by default.
Ntopng web interface is accessible:
- from specified TCP port (default is
3000
with access only from green interfaces) - from Server Manager using a proxypass:
https://<server>:980/<alias>
(seealias
prop)
The software is configured to use a minimal redis instance named redis-ntopng.
Database Reference¶
Properties:
Authentication
: can beenabled
ordisabled
. Default isdisabled
Interfaces
: comma-separeted list of monitored interfacesalias
: auto-generated alias to be used as proxypass URL on httpd-admin
ntopng=service
Authentication=disabled
Interfaces=br0
TCPPort=3000
access=green
alias=4de50f46b8d3e5ec829aba759431b48ad8173768
status=enabled
Download lates upstream release¶
To download directly from stable repository:
wget -r --no-parent -A n2n*rpm -A ndpi*rpm -A ntopng*rpm -A pfring*rpm http://packages.ntop.org/centos/7/x86_64/Packages/
To access the stable repository:
cd /etc/yum.repos.d/
wget http://packages.ntop.org/centos-stable/ntop.repo -O ntop.repo
Links¶
- Official site: http://www.ntop.org/products/ntop/
nethserver-ntp¶
Manage server date and time, ntp server for lan clients.
How it works¶
When the time and date are modified, some services must be restarted. Each package must subscribe service2adjust
interface. Note that there’s no need to restart rsyslog on date and time changing. Rsyslog daemon is smart enough to read new time setting on the fly.
Date and time can be set manually or automatically via a ntp server.
The current time zone is accessible from the configuration database, but the value is read and written using systemd commands.
Events¶
nethserver-ntp-update
, standard event triggered on package installation/update.nethserver-ntp-save
, standard event triggered on UI moduleSAVE
action
Actions¶
nethserver-ntp-localtime
, create a lint to the zoneinfo file from/usr/share/zoneinfo
to/etc/localtime
according toTimeZone
key in _configuration_ database.nethserver-ntp-clock-adjust
, adjust the system date and hardware clock when NTP is disabled. Requires the event name and timestamp arguments. When NTP is enabled, restarts the ntp daemon.
Daemon status¶
Run following command command:
chronyc sourcestats
Sample output:
210 Number of sources = 1
Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev
==============================================================================
yourntp.server.com 41 20 43m +0.001 0.172 +58ns 227us
nethserver-nut¶
The nethserver-nut package provides: * Network UPS Tool (NUT) configuration * web UI for simple configuration * optional collectd support
Configuration¶
The package provides two configuration modes:
- Master: a master node is a machine directly connected to the UPS port
- Slave: a slave node is a machine connected to the master via tcp
If you need stand-alone mode, just configure the machine as master.
Master node¶
A master node receives events from UPS, sends events to slaves and takes decisions (eg. power off the machine).
DB properties:
- Mode: master
- Device: UPS port, can be a device like
/dev/ttyS0
orauto
if the UPS is USB - Model: UPS driver name selected from
/usr/share/driver.list
- Description: UPS model description like “APC - Smart-UPS USB (**)” (used only on web UI)
- Password: chosen password to communicate between master and slaves
- Ups: ups name, default is
UPS
- User: upsmon user name, default is
upsmon
- Notify: if
enabled
notifications about UPS events are sent to the admin user, default isdisabled
The password is generated during first installation from the nethserver-nut-conf
action.
Slave node¶
A slave node receives events from a master and takes decisions (eg. power off the machine).
DB properties:
- Mode: slave
- Master: host name or ip address of master node
- Password: chosen password to communicate between master and slaves
- User: upsmon user name, default is
upsmon
Collectd-nut¶
If server is in master mode, collectd is configured to monitor the local connected UPS.
Optional package required:
- collectd-nut
- nethserver-collectd
nethserver-openssh¶
OpenSSH is the premier connectivity tool for remote login with the SSH protocol. It encrypts all traffic to eliminate eavesdropping, connection hijacking, and other attacks. In addition, OpenSSH provides a large suite of secure tunneling capabilities, several authentication methods, and sophisticated configuration options.
The OpenSSH suite consists of the following tools:
- Remote operations are done using ssh, scp, and sftp.
- Key management with ssh-add, ssh-keysign, ssh-keyscan, and ssh-keygen.
- The service side consists of sshd, sftp-server, and ssh-agent.
Properties¶
the props of the service to configure openssh:
sshd=service
AllowGroups=group1,group2:sftp,group3
AllowLocalGroups=wheel,adm
LoginGraceTime=2m
MaxAuthTries=6
PasswordAuthentication=yes
PermitRootLogin=yes
Protocol=2
SubsystemSftp=yes
TCPPort=22
UsePAM=yes
access=green,red
status=enabled
StrongEncryption=enabled
AllowGroups
: a comma separated list of groups from the accounts provider, allowed to connect with SSH. If the option:sftp
is specified then the group is restricted to SFTP. This prop is considered if the property$sssd{'ShellOverrideStatus'}
is enabled.AllowLocalGroups
: a comma separated list of group names in/etc/groups
that are granted full SSH and SFTP access.SubsystemSftp
: (yes|no) enable the sftp serviceLoginGraceTime
: The time after which the server disconnects if the user has not successfully logged in.MaxAuthTries
: Specifies the maximum number of authentication attempts permitted per connection. Once the number of failures reaches half this value, additional failures are logged.PasswordAuthentication
: (yes,no) Specifies whether password authentication is allowed.PermitRootLogin
: (yes,no) Specifies whether root can log in using ssh.TCPPort
: Use this TCP port to use sshdUsePAM
: (yes,no) Pam can be used to authentify useraccess
: Allow ssh connection to your firewall following different zones.status
: (enabled,disabled) Enable or disable the sshd serviceStrongEncryption
: (enabled,disabled) Enable or disable the strong encryption by requiring strong ciphers, HostKeyAlgorithms, KexAlgorithms, MACs. Old clients might not be able to connect to the server once enabled.
Events¶
Two events takes care to expand and restart the service
nethserver-openssh-update
(sshd is restarted)
nethserver-openssh-save
(sshd is reloaded)
One time password (2FA)¶
2FA is available only for password authentication. Authentication with public keys will not be affected by 2FA.
The Openssh server can be protected by a Two-factor authentication (2FA). It adds an extra layer of security required to access the new Server Manager. First, users will enter user name and password, then they will be required to provide a temporary verification code generated by an application running on their smartphone.
2FA is disabled by default. Each user can enable it by accessing the Two-factor authentication section under Settings page.
nethserver-openvpn¶
Supported VPN are:
- Client to server (roadwarrior)
- Network to network (net2net)
Events¶
openvpn-tunnel-create
: fired when a new tunnel is created, takes the tunnel name as argumentopenvpn-tunnel-delete
: fired when a new tunnel is deleted, takes the tunnel name as argumentopenvpn-tunnel-modify
: fired when a new tunnel is modified, takes the tunnel name as argumentnethserver-openvpn-save
: fired when roadwarrior account or server is changedopenvpn-tunnel-upload
: used to transform a given file into a ready-to-use tunnel client
Roadwarrior accounts¶
Accounts are used to identify clients connecting to the server itself. There are two types of accounts:
- user account: system user with VPN access using user name and password
- vpn-only account: simple account with VPN access only
Each account can be used in a roadwarrior connection (host to net).
If a net to net tunnel is needed, VPNRemoteNetwork
and VPNRemoteNetmask
properties must be set to inform the server about remote routes.
When a new account is created, a certificate with same name is generated inside /var/lib/nethserver/certs
directory.
Properties:
VPNRemoteNetwork
: remote networkVPNRemoteNetmask
: remote netmaskOpenVpnIp
: reserved IP for the client
Roadwarrior server¶
Certificates¶
All certificates are signed using NethServer default RSA key (/etc/pki/tls/private/NSRV.key
).
CA environment¶
CA configuration is stored inside /var/lib/nethserver/
directory, all certificates are stored inside /var/lib/nethserver/certs
. The nethserver-openvpn-conf
action creates:
serial
,certindex.attr
and/certindex
: database of valid and revoked certificatescrlnumber
and/etc/openvpn/certs/crl.pem
: certificate revocation listdh1024.pem
: key for TLS negotation
Certificate creation¶
Certificates in PEM format can be created using the command:
/usr/libexec/nethserver/pki-vpn-gencert <commonName>
The commonName
parameter is an unique name stored inside the certificate.
The command will generate key
, crt
and csr
file.
Each generated certificate is referred with a numeric id and saved inside certindex
database. OpenSSL will also create a certificate named using the generated id (eg. 02.pem
).
Certificate revocation¶
Certificate revocation is done via the command:
/usr/libexec/nethserver/pki-vpn-revoke [-d] <commonName>
The commonName
parameter is an unique name stored inside the certificate.
If ‘-d’ option is enabled, also delete crt
, csr
, pem
and key files
Certificate renewal¶
All certificates will expire after X
days, where X
is the value of CertificateDuration
property inside pki
key.
Renew is done via the command:
/usr/libexec/nethserver/pki-vpn-renew <commonName>
The commonName
parameter is an unique name stored inside the certificate.
Client configuration is generated using /usr/libexec/nethserver/openvpn-local-client
command.
The file will contain the CA certificate inside the <ca>.
Example:
/usr/libexec/nethserver/openvpn-local-client myuser
The OpenVPN server listens on a management socket: /var/spool/openvpn/host-to-net
.
It’s possible to retrieve server status and execute commands using the socket.
Available scripts:
/usr/libexec/nethserver/openvpn-status
: retrieve status of connected clients and return result in JSON format/usr/libexec/nethserver/openvpn-kill
: kill a connected client, exits 0 on success, 1 otherwise
Example with netcat:
>INFO:OpenVPN Management Interface Version 1 -- type 'help' for more info
status
OpenVPN CLIENT LIST
Updated,Thu Jan 23 09:22:24 2014
Common Name,Real Address,Bytes Received,Bytes Sent,Connected Since
ROUTING TABLE
Virtual Address,Common Name,Real Address,Last Ref
GLOBAL STATS
Max bcast/mcast queue length,0
END
See more on management options: http://openvpn.net/index.php/open-source/documentation/miscellaneous/79-management-interface.html
Configuration database¶
Properties:
status
: enable or disable the OpenVPN server, can beenabled
ordisabled
, default isdisabled
AuthMode
: authentication mode, can bepassword
,certificate
orpassword-certificate
UDPPort
: server listen port, default is1194
Mode
: network mode, can berouted
orbridged
. Default isrouted
.ClientToClient
: can beenabled
ordisabled
, default isdisabled
. When enabled, traffic between VPN clients is allowedCompression
: can beenabled
,disabled
or a valid compression alghoritm (see man openvpn). If set to enabled, adaptive LZO compression is usedRemote
: comma-separated list of IPs or host names, it’s used as multiple remote option inside client configuration generation scriptPushDns
: if set, push the specified DNS as DHCP optionPushDomain
: if set, push the specified domain as DHCP optionPushExtraRoutes
: ifdisabled
, only routes for green interface are pushed, if enabled also all static routes will be pushed. Default isenabled
PushNbdd
: if set, push the specified NBDD as DHCP optionPushWins
: if set, push the specified WINS as DHCP optionCustomRoutes
: a comma separated listed of CIDR to be pushed as extra routes to VPN clientsTopology
: set roadwarrior server topology, for valid values seeman openvpn
NotifyStatus
: can beenabled
ordisabled
, default isdisabled
. Ifenabled
, send a mail notification toNotifyAddresses
upon roadwarrior user connect or disconnectNotifyAddresses
: comma-separated list of mail addresses, destinations forNotifyStatus
property
If mode is bridged
:
BridgeEndIP
: first client IP pool, must be inside the LAN range and outside DHCP rangeBridgeStartIP
: last client IP pool, must be inside the LAN range and outside DHCP rangeBridgeName
: name of the bridge, default isbr0
TapInterface
: name of bridged tap interface, default istap0
If mode is routed
:
Network
: network of VPN clients, eg. 192.168.6.0Netmask
: netmask of VPN clients, eg. 255.255.255.0RouteToVPN
: can beenabled
ordisabled
, default isdisabled
. When enabled, all traffic from client will be routed via VPN tunnel
Reference¶
Example:
openvpn@host-to-net=service
AuthMode=password
BridgeEndIP=192.168.1.122
BridgeName=br0
BridgeStartIP=192.168.1.121
ClientToClient=disabled
Mode=routed
Netmask=255.255.255.0
Network=192.168.6.0
NotifyAddresses=root@localhost
NotifyStatus=enabled
PushDns=
PushDomain=
PushExtraRoutes=enabled
PushNbdd=
PushWins=
Remote=
RouteToVPN=disabled
TapInterfaces=tap0
Topology=subnet
UDPPort=1194
access=green,red
status=enabled
Tunnel topology¶
Available topologies are subnet
and p2p
If topology is p2p
:
- only
psk
authentication mode is available - options
LocalPeerIP
andRemotePeerIP
must be set for both server and client - options
RemoteNetworks
must be set if the client should be able to reach networks behind the tunnel server
If topology is subnet
:
- only
password
,certificate
orpassword-certificate
authentication mode are available - options
Network
must be set for the server
Tunnel clients¶
OpenVPN clients are used to connect to a tunnel server obtaining a net2net VPN.
Common properties:
AuthMode
: default value iscertificate
. Possible values:certificate
: use x509 certificate. Certificates, including CA and private key, are saved in/var/lib/nethserver/certs/clients
directory in a PEM file namedkey
.pempassword
: use user name and passwordpassword-certificate
: use user name, password and a valid x509 certificatepsk
: use a pre-shared key
Cipher
: a valid OpenVPN cipher amongopenvpn --show-ciphers
Compression
: can beenabled
,disabled
or a valid compression alghoritm (see man openvpn). If set to enabled, adaptive LZO compression is usedLocalPeerIP
andRemotePeerIP
: IP for the Point to Point hostsPassword
: password used for authentication, ifAuthMode
ispassword
orpassword-certificate
Protocol
: can beudp
ortcp
, default isudp
RemoteHost
: a list of remote server hostnames or ip addressesRemoteNetworks
: list of networks in CIDR format, for each network will be created a route. This networks will also be used by the firewall library to calculate the zone of VPN hosts used inside the firewall rules.RemotePort
: remote host portUser
: username used for authentication, ifAuthMode
ispassword
orpassword-certificate
WanPriorities
: an ordered list of red interfaces which will be used to connect to the server, can be used to prefer a faster WAN over a slower oneTopology
: can besubnet
(default) orp2p
status
: enable or disable the OpenVPN server, can beenabled
ordisabled
, default isenabled
Files:
- pre-shared key:
/var/lib/nethserver/certs/clients/<name>.key
- certificates:
/var/lib/nethserver/certs/<name>.{pem,key}
Database reference¶
Database: vpn
c1=tunnel
AuthMode=psk
Cipher=
Compression=disabled
Crt=
Mode=routed
Protocol=udp
Psk=
RemoteHost=1.2.3.4,8.8.6.7
RemotePort=1122
Topology=subnet
WanPriorities=eth2,eth1
status=enabled
clntnh2d=tunnel
AuthMode=certificate
Cipher=
Compression=enabled
Mode=routed
Protocol=udp
RemoteHost=1.2.3.4,8.8.6.7
RemotePort=1244
Topology=subnet
status=enabled
Tunnel servers¶
Servers are instance of OpenVPN listening for incoming connections. Each server runs on its own port and can handle many clients.
When a server is created the following files will be generated:
- a certificate and a private key:
/var/lib/nethserver/openvpn-tunnels/<name>.crt
and/var/lib/nethserver/openvpn-tunnels/keys/<name>.key
- a pre-shared key
/var/lib/nethserver/openvpn-tunnels/<name>.key
Certificate generation uses a custom version of easy-rsa, see /usr/share/nethserver-openvpn
directory content.
Properties:
Cipher
: a valid OpenVPN cipher amongopenvpn --show-ciphers
Compression
: can beenabled
,disabled
or a valid compression alghoritm (see man openvpn). If set to enabled, adaptive LZO compression is usedLocalNetworks
: list of networks in CIDR format, each network will be pushed as route to the clientLocalPeerIP
andRemotePeerIP
: IP for the Point to Point hostsNetwork
: network address of the VPN tunnelPort
: listen portProtocol
: can beudp
ortcp
, default isudp
PublicAddresses
: list of public IPs or host names used by clients to connect to the serverRemoteNetworks
: list of networks in CIDR format, for each network will be created a local routeTopology
: can besubnet
(default) orp2p
status
: enable or disable the OpenVPN server, can beenabled
ordisabled
, default isdisabled
Database reference¶
Database: vpn
server1=openvpn-tunnel-server
Cipher=
Compression=enabled
LocalNetworks=192.168.1.0/24
RemotelNetworks=192.168.6.0/24
Network=10.10.11.0/24
Port=1282
Protocol=udp
PublicAddresses=1.2.3.4,test.local.neth.eu
Topology=subnet
status=enabled
psk1=openvpn-tunnel-server
Cipher=
Compression=enabled
LocalNetworks=10.132.0.0/16
LocalPeer=10.68.218.1
Port=1248
Protocol=udp
PublicAddresses=1.2.3.4,myfw.host.org
RemotePeer=10.68.218.2
Topology=p2p
status=disabled
Templates¶
The main templates for the configuration are:
/etc/e-smith/templates/etc/openvpn/host-to-net.conf
: allows the customization of roadwarrior server/etc/e-smith/templates/openvpn-tunnel-server/
: allows the customization of tunnel server, it’s expanded in/etc/openvpn/<tunnel_server_name>.conf
/etc/e-smith/templates/openvpn-tunnel-client/
: allows the customization of tunnel client, it’s expanded in/etc/openvpn/<tunnel_client_name>.conf
Log files and systemd instances¶
All OpenVPN services are handled using systemd instances of the openvpn
service.
Every instance has the following form: openvpn@<instance_name>
where
instance_name
is name of the key inside the vpn
e-smith db.
The roadwarrior server has a fixed name: host-to-net
.
Instances can be inspected using systemctl
command:
systemctl status openvpn@host-to-net
systemctl status openvpn@server1
The roadwarrior logs can be found here:
/var/log/openvpn/host-to-net-status.log
/var/log/openvpn/openvpn.log
The log of each OpenVPN instance can be seen using journalctl
command.
Example:
journalctl -u openvpn@client1
journalctl -u openvpn@host-to-net
nethserver-phonenehome¶
This tool is used to tracks all NethServer’s installations around the world.
- Use API to get and set installation
- Visualize the installations in Google Maps with nice markers
Overview¶
This tool is composed by two parts: server and client and the data are stored and read through REST APIs. The REST APIs can be found in index.php
file. To send data to REST APIs check out the phone-home
file in root/etc/cron.weekly
.
Configuration¶
After the repository clonig, you must set the correct placeholders in certain files.
Server¶
config.php¶
Line 2
change __dbhost__, __dbuser__, __dbpass__, __dbname__
with your own credentials:
$GLOBALS['$dbhost'] = "__dbhost__";
$GLOBALS['$dbuser'] = "__dbuser__";
$GLOBALS['$dbpass'] = "__dbpass__";
$GLOBALS['$dbname'] = "__dbname__";
widget_map.html¶
Line 53
change __serverip__
with the server ip used to host REST APIs:
// ip server with api
var server_ip = '__serverip__';
Create table in your database¶
After setting up your own credentials, simply run (MySQL syntax):
CREATE DATABASE phonehome;
GRANT ALL ON phonehome.* TO user IDENTIFIED BY 'password';
CREATE TABLE IF NOT EXISTS phone_home_tb (
uuid VARCHAR(40) PRIMARY KEY,
release_tag VARCHAR(10) NOT NULL,
ip VARCHAR(16) NOT NULL,
country_code VARCHAR(5),
country_name VARCHAR(40),
country_location_lat VARCHAR(40),
country_location_lng VARCHAR(40),
reg_date TIMESTAMP
)
Client¶
phone-home¶
Create a file named phone-home
in /etc/sysconfig
and set the correct infos:
SERVER_IP=__serverip__
PROXY_SERVER=__proxyserver__
PROXY_USER=__proxyuser__
PROXY_PASS=__proxypass__
PROXY_PORT=__proxyport__
the variables PROXY_SERVER, PROXY_USER, PROXY_PASS, PROXY_PORT
are not mandatory.
nethserver-postgres¶
PostgreSQL is a database server and it listens on standard port 5432.
Access policy¶
From localhost:
- the postgres user can access without password
- all other users must use password authentication
From any other network:
- all users must use password authentication
Example for accessing with postgres user:
su - postgres
psql
Backup¶
Backup and restore is not implemented system-wide: every application should provide its own scripts.
Backup actions must be linked inside the pre-backup-data
event.
Example of backup action:
#!/bin/bash
su - postgres -c "pg_dump myapp > /var/lib/nethserver/myapp/myapp.sql"
Restore actions must be linked inside the post-restore-data
event.
Example of restore action:
#!/bin/bash
if [ -f /var/lib/nethserver/myapp/myapp.sql ]; then
drop_sql=`mktemp`
chown postgres:postgres $drop_sql
# drop all existing connections to the db and block new ones
echo "UPDATE pg_database SET datallowconn = 'false' WHERE datname = 'myapp';" >> $drop_sql
echo "SELECT pg_terminate_backend(procpid) FROM pg_stat_activity WHERE datname = 'myapp';" >> $drop_sql
# drop the db, then recreate it
echo "DROP DATABASE myapp;" >> $drop_sql
password=`perl -e "use NethServer::Password; print NethServer::Password::store('myapp');"`
echo "CREATE database myapp; CREATE USER sonicle WITH PASSWORD '$password'; GRANT ALL PRIVILEGES ON DATABASE myapp to myuser;" >> $drop_sql
# allow new connections to db
echo "UPDATE pg_database SET datallowconn = 'true' WHERE datname = 'myapp';" >> $drop_sql
su - postgres -c "psql < $drop_sql"
su - postgres -c "psql myapp < /var/lib/nethserver/myapp/myapp.sql"
rm -f $drop_sql
fi
nethserver-pulledpork¶
Pulledpork it’s a rule management tool for Suricata (and Snort). Actual configuration downloads only Emerging Threats.net Open rulesets.
After the download, multiple files are created inside /etc/suricata/rules
directory.
Official site: https://github.com/shirkdog/pulledpork
nethserver-release¶
This RPM can be installed on a plain CentOS 7 minimal. It provides the
nethserver-install
command that downloads, installs and configures
additional YUM repositories and RPMs for the NethServer system.
Building a release RPM¶
- The Version tag is fixed to “7”. Increment the .spec file Release tag (e.g.: “12%{?dist}”)
- Write the %changelog entry in the .spec file
- Commit the above changes
- Create a git tag like “7r12”. Do not use any “-” (minus) sign as separator!
- Push the tag and the commit to start the automated build on Travis CI
Builds started from a tagged commit are published to “updates”!
More information: https://docs.nethserver.org/projects/nethserver-devel/en/v7/building_rpms.html
nethserver-rh-php56-php-fpm¶
Configure default installation of PHP 5.6 running on FPM.
The default configuration of FPM is from upstream and can freely change
(no template): /etc/opt/rh/rh-php56/php-fpm.d/www.conf
.
Adding new configuration¶
If you need a new configuration, simply drop a file inside /etc/opt/rh/rh-php56/php-fpm.d/
directory and execute:
signal-event nethserver-rh-php56-php-fpm-update
nethserver-rh-php71-php-fpm¶
Configure default installation of PHP 7.1 running on FPM.
The default configuration of FPM is from upstream and has
been customized to listen on port 9001 (template: /etc/opt/rh/rh-php71/php-fpm.d/z_nethserver.conf
).
Adding new configuration¶
If you need a new configuration, simply drop a file inside /etc/opt/rh/rh-php71/php-fpm.d/
directory and execute:
signal-event nethserver-rh-php71-php-fpm-update
nethserver-roundcubemail¶
Roundcube is a fast webmail client written in PHP.
Database¶
Configuration is saved in roundcubemail
key inside configuration
database.
Available properites:
Server
: server address of the mail server, default islocalhost
access
: can bepublic
orprivate
, default ispublic
- public: webmail can be accessed from any networks
- private: webmail can be accessed only from green interfaces and trusted networks
PluginsList
: comma separated list of enabled plugins, default ismanagesieve,markasjunk
. Before adding an option to this property, please be sure the plugin is already installed. A list of bundled plugins can be found inside file:/usr/share/roundcubemail/plugins
directory.skin
: default islarry
other skins can be found in/usr/share/roundcubemail/skins
Example:
roundcubemail=configuration
PluginsList=managesieve,markasjunk
Server=localhost
access=private
skin=larry
Configuration can be applied using the nethserver-roundcubemail-update
event.
nethserver-samba¶
File and print server for a MS-Windows network based on Samba.
Configuration database¶
Example:
smb=service
...
NetbiosAliasList=
DeadTime=10080
WinsServerStatus=disabled
WinsServerIP=
UseCups=enabled
UseClientDriver=yes
HomeAdmStatus=disabled
ShareAdmStatus=disabled
InheritOwner=yes
NetbiosAliasList
Seenetbios aliases
parameter in smb.conf(5) manpage.DeadTime
(days) Seedeadtime
parameter in smb.conf(5) manpage.WinsServerStatus
ifenabled
act as a WINS server.WinsServerIP
ipaddress ifWinsServerStatus
isdisabled
,nmbd
will register with the given WINS server. Seewins server
,remote announce
,remote browse sync
parameters in smb.conf(5) manpage.UseCups {enabled,disabled}
Use cups as printing server.UseClientDriver {yes,no}
Seeuse client driver
parameter in smb.conf(5) manpage.HomeAdmStatus {enabled,disabled}
Enable home$ hidden, administrative shareShareAdmStatus {enabled,disabled}
Setadmin users = "@domain admins"
on every shared folderInheritOwner {yes,no}
Mapped toinherit owner
property of smb.conf
Accounts database¶
Only records with type ibay
.
Properties:
SmbStatus
ifenabled
, activates ibay sharing through SMB protocolSmbProfileType
select the profile template to apply to the share (optional).- The template path must be placed into
/etc/e-smith/templates/etc/smb.conf/
and prefixed byibay-
. Eg:default
profile is located at/etc/e-smith/templates/etc/smb.conf/ibay-default
.
SmbRecycleBinStatus
: enable or disable the recycle bin; when a file is deleted it is moved inside the recycle bin.SmbShareBrowseable
: controls the visibility of the shared folder, default isenabled
.
Example:
iba1=ibay
AclRead=domadmins,admin
AclWrite=domadmins,admin
Description=test
GroupAccess=rw
OtherAccess=r
OwningGroup=locals
SmbGuestAccessType=none
SmbRecycleBinStatus=disabled
SmbShareBrowseable=enabled
SmbStatus=enabled
nethserver-samba-audit¶
Samba audit is a simple audit module for samba which uses vfs_full_audit module. All operations are logged in a file and a logrotate job parses all entries and store it into a MySQL db. Logs are browseable using web interface.
SambaAudit is based on a Samb Audit project. See: http://sourceforge.net/projects/smbdaudit/ Current implementation uses standard full_audit vfs module instead of mysql_audit.
Configuration¶
The package configures Samba standard vfs audit to log write and read actions inside the /var/log/smbaudit.log
file.
Every night, a script called by logrotate
parses the log file and puts all audit actions inside the smbaudit
MySQL database.
The database can be explored using Cockpit or the legacy PHP UI.
The packages adds the following properties to the smb
key:
AuditAlias
: auto-generate alias to access the legacy UIAuditLogRead
: can beenabled
ordisabled
. Ifenabled
read actions are stored in the database during the parsing, ifdisabled
only write actions will be written to the database.
Example:
smb=service
AuditAlias=43d5xxxxxxxxxxxxxxxxf023e46a11a4b7cb233a
AuditLogRead=disabled
DeadTime=10080
HomeAdmStatus=disabled
InheritOwner=no
NetbiosAliasList=
ShareAdmStatus=disabled
TCPPorts=139,445
UseClientDriver=yes
UseCups=enabled
WinsServerIP=
access=green
status=enabled
To enable the audit for a shared folder, use the SmbAuditStatus
property to the ibay
record.
Example:
test=ibay
...
SmbAuditStatus=enabled
...
nethserver-smartd¶
Configure smartd daemon to start at boot. smartd is a daemon that monitors the Self-Monitoring, Analysis and Reporting Technology (SMART)
Database:
smartd=service
status=enabled
nethserver-squid¶
This package configure the well-known Squid web proxy.
Squid rpms are from upstream.
Configuration¶
All properties are saved in the squid
key under the configuration
database.
Properties:
- BlueMode: change Squid operation mode on blue networks. It has same values and defaults of
GreenMode
- BypassDomains: comma separeted list of domains bypassed when the proxy is set in transparent mode
- DiskCache: disabled by default, if enabled it actives the disk caching system for squid
- DiskCacheSize: maximum value of squid cache on disk
- GreenMode: change Squid operation mode on green networks.
Can be:
manual
,authenticated
,transparent
,transparent_ssl
. Default is:manual
- KrbPrimaryList: name for Kerberos keytab (used for Active Directory integration)
- KrbStatus: if set to enabled a ticket credential cache file is kept valid by the hourly cron job (used for Active Directory integration)
- MaxObjSize: objects larger than this setting will not be saved on disk. If speed is more desirable than saving bandwidth, this should be set to a low value
- MemCacheSize: value of squid cache on memory
- MinObjSize: can be left at 0 to cache everything, but may be raised if small objects are not desired in the cache.
- NoCache: comma separated list of domains which will be not cached
- ParentProxy: in the form host:port, if omitted port is default to 3128. Default is empty
- PortBlock: if enabled, block port 80 and 443. Default is:
disabled
- SafePorts: comma separated list of ports thath can be accessed through the proxy. Listed ports will be added to the default list of safe and ssl ports
Database example¶
Example:
squid=service
BlueMode=manual
DiskCache=disabled
DiskCacheSize=100
GreenMode=transparent
KrbPrimaryList=HTTP
KrbStatus=enabled
MaxObjSize=4096
MemCacheSize=256
MinObjSize=1
NoCache=www.nethserver.org
ParentProxy=
PortBlock=disabled
TCPPorts=3128,3129,3130
access=private
status=enabled
Transparent mode¶
When the proxy is enabled in transparent mode, all traffic destined to the port 80 is redirect to the Squid (port 3129). This configuration requires Shorewall.
SSL peek and splice¶
If the proxy is enabled in transparent SSL mode, also all traffic destined to port 443 is redirected to Squid (port 3130). The daemon does not inspect SSL traffic, but visited sites can be processed using the web filter.
Known bugs¶
You could find this kind of errors inside /var/log/squid/cache.log
2016/12/09 09:44:18 kid1| SECURITY ALERT: on URL: avatars0.githubusercontent.com:443
2016/12/09 09:44:18 kid1| SECURITY ALERT: Host header forgery detected on local=151.101.60.133:443 remote=192.168.5.22:40950 FD 166 flags=33 (local IP does not match any domain IP)
In this case, when accessing github, the avatars won’t be displayed by the browser, and you can find a “Timeout error” for the not loaded images.
This kind of errors can’t be fixed. See official documentation for workarounds:
Authenticated mode¶
Authentication schema depends on system configuration:
- standard authentication for system users is done over LDAP
- if Samba AD is installed, clients can use Kerberos (SPNEGO/GSSAPI)
Bypasses¶
The implementation supports 3 kind of bypass:
- source bypass
- destination bypass
- domains bypass
Source and destination bypass¶
Bypass rules are saved inside the fwrules
databases.
A bypass can be of two types:
- bypass-src: listed origin host are bypassed
- bypass-dst: listed target host are bypassed
Properties:
- Host: a host object, like a remote or local host
- status: can be
enabled
ordisabled
- Description: optional description
Bypass example:
boss=bypass-src
Description=Boss without proxy
Host=host;bosspc
status=enabled
Domains bypass¶
All requests to domains listed inside the BypassDomains
property will not
be redirect to the transparent proxy.
The implementation uses the ipset feature of Dnsmasq.
Each time a listed domain is accessed from the client, Dnsmasq resolves the IP
and add it to squid-bypass
ipset.
The squid-bypass
ipset is then used as exception inside Shorewall REDIRECT rule.
Notes:
- all clients must use the server as DNS
- Dnsmasq name resolution works for the listed domains and all sub-domains
Cache¶
There is an event called nethserver-squid-clear-cache
that empties the cache.
Priority and divert rules¶
The squid
database contains multiple records of type rule
.
Each rule has following properties:
key
: it’s a numeric uniq idAction
:priority;low
: add low priority packet marker usingtcp_outgoing_mark
priority;high
: add high priority packet marker usingtcp_outgoing_mark
provider;<name>
: add packet marker for provider<name>
usingtcp_outgoing_mark
force;<name>
: force output traffic to<name>
provider usingtcp_outgoing_address
Description
: optional descriptionDst
: comma-separeted list of domains, this is converted to adstdomain
ACLSrc
: firewall object, supported objects are: role, host, zone, ip range and cidr . This is converted tosrc
ACLstatus
: can beenabled
ordisabled
Example:
1=rule
Action=priority;low
Description=
Dst=yahoo.com
Src=host;giacomo
status=enabled
2=rule
Action=provider;fast
Description=
Dst=nethserver.org,nethesis.it
Src=host;birro
status=enabled
3=rule
Action=force;slow
Description=
Dst=
Src=cidr;cidr1
status=enabled
WPAD¶
WPAD is located at /var/www/html/wpad.dat
.
The web server is configured to allow the download only from trusted and blue networks,
but be aware that you need to manually open the httpd port for blue networks (see Add a new service).
The WPAD returns:
- DIRECT, if squid is disabled or the requesting client is inside a network where the proxy is configured in transparent mode
- IP of corresponding network interface, if the requesting client is inside a network where the proxy is configured in manual or authenticated mode
- proxy.<domain>, if the server is joined to Active Directory and the requesting client is inside a network where the proxy is configured in manual or authenticated mode
Also WPAD file includes all source and destination bypasses.
Miscellaneous options¶
The following options are always enabled:
- buffered logs
- SNMP support on port 3401
nethserver-squidguard¶
The package configures ufdbGuard, a URL filter for squid. The configuration is based on profiles. Each profile is composed by:
- a user, group of users, host or group of hosts accessing a web page
- a filter which includes allowed and denied sites
- a time frame in which the filter is active
The system comes with a default profile which is applied to any host/user who doesn’t fit on a specific profile.
Features of ufdbGuard:
- 3-4 times faster than squidGuard
- a multithreaded daemon with one copy of the database in memory
- detects HTTPS proxy tunnels
- detects SSH-based tunnels
- blocks HTTPS for URLs without FQDN
- blocks HTTPS for sites without a properly signed SSL certificate
- uses in-memory databases
- enforce the SafeSearch feature on Google and other search engines
- a test mode (-T option) allows you to test a URL filter database without actually blocking sites
For the block page on th server, see https://github.com/NethServer/blocked.nethserver.org/.
Inner workings¶
For each request, squid sends the URL to one ufdbgclient
redirector (which runs as user squid),
which in turn asks to the ufdb
daemon (which runs as user ufdb).
Known limitations¶
- Transparent URL filtering on HTTPS websites can only block whole domains, because ufdbGuard can only receive
the domain name, not the full URL
- Redirected HTTPS show an error instead of the block page
Blacklists¶
Blacklists are updated every night using the script: /etc/cron.daily/update-squidguard-blacklists
The script will download and merge all blacklists listed in /etc/squid/blacklists
. This actions can take several minutes.
Databases¶
The package uses the squidguard
key inside the configuration
database, also it creates a new contentfilter
database
for profiles and filters configuration.
configuration¶
The key squidguard
contains all settings. Properties are:
- BlockedFileTypes: comma separated list of blocked file extensions
- CustomListURL: URL to download a custom blacklist. The blacklist must follow SquidGuard standard
- DomainBlacklist: comma separated domain list, this domains are always blocked
- DomainWhitelist: comma separated domain list, this domains are always allowed
- Expressions: if enabled, allow regular expression on blacklists categories. Can be
enabled
ordisabled
, default isdisabled
- IdleChildren: minimum number of idle processes. Default is 5
- Lists: comma separated list of blacklist names. Possible values are:
toulouse
andcustom
. If set tocustom
, make sureCustomListURL
is set. - MaxChildren: maximum number of processes. Default is 20
- RedirectUrl: custom URL for block page. See Block page
- StartupChildren: minimum number of process children on startup. Default is 5
Note
Modifying following parameters can greatly affect memory usage: IdleChildren, MaxChildren, StartupChildren
Example:
squidguard=configuration
BlockedFileTypes=
CustomListURL=
DomainBlacklist=microsoft.com
DomainWhitelist=nethserver.org,nethesis.it
Expressions=disabled
IdleChildren=5
Lists=toulouse
MaxChildren=20
RedirectUrl=
StartupChildren=5
UrlBlacklist=
UrlWhitelist=
status=enabled
The service key ufdbGuard
has enabled/disabled state
according to the status prop of the squidguard
(see nethserver-squidguard-ufdb-status action).
contentfilter¶
The contentfilter
database can contain three kind of records:
- category: a custom categorized list of domains blocked or allowed. Custom categories can be added to a filter
- filter: an object describing which categories must be blocked or allowed
- time: when the filter must be applied, it contains week days and time
- profile: a link between the above objects describing WHO (host or user), WHAT (filter) and WHEN (time)
Categories¶
Properties:
- Domains: comma separated list of domains
- Description: optional description
Category example:
mycategory=category
Description=My Category
Domains=nethesis.it,nethserver.org
Filters¶
Properties:
- BlackList: enable or disable the global blacklist (
DomainBlacklist
). Can beenabled
ordisabled
- BlockAll: can be
enabled
ordisabled
. If disabled, all categories listed inCategories
are blocked and all other sites are allowed. If enabled, all listed categories inCategories
are allowed and all other sites are blocked - BlockFileTypes: enable or disable the global file extension list (
BlockedFileTypes
). Can beenabled
ordisabled
- BlockIpAccess: if enabled, sites can be accessed only using a domain name (not an IP address). Can be
enabled
ordisabled
- Categories: comma separated list of categories blocked or allowed. If a category is not present inside the SquidGuard db (
/var/squidGuard/Blacklists
), the category will be excluded from configuration file to avoid ufdbGuard panic-mode (no filter) - Description: optional description
- WhiteList: enable or disable the global whitelist (
DomainWhitelist
. Can beenabled
ordisabled
- Removable: can be
yes
orno
. If set tono
the record can’t be removed from web interface
Filter example:
myfilter=filter
BlackList=enabled
BlockAll=disabled
BlockFileTypes=disabled
BlockIpAccess=disabled
Categories=aggressive,alcohol,weapons,warez
Description=Default filter
WhiteList=enabled
Times¶
Properties:
- Days: comma separated list of week days. Valid values are:
- m: Monday
- t: Tuesday
- w: Wednesday
- h: Thursday
- f: Friday
- a: Saturday
- s: Sunday
- Description: optional description
- EndTime: hour of the day in 24h format or empty
- StartTime: our of the day in 24h format or empty
Time example:
worktime=time
Days=m,t,w,h,f
Description=Work time
EndTime=18:30
StartTime=08:30
Profiles¶
Properties:
- Filter: a filter object
- *FilterElse: an optional filter object, applied when none of the referenced Time rules apply
- Src: it can be an object of type user, user group, host, host group, zone or role. Otherwise, if it is a string, the system will assume the profile is associated with an user from Active Directory; the system must be joined to a domain
- Time: a CSV list time object references (optional)
- Description: optional description
- Removable: can be
yes
orno
. If set tono
the record can’t be removed from web interface
Profile example:
myprofile=profile
Description=My profile
Filter=filter;badboys
FilterElse=filter;enjoy
Src=host;demo
Time=time;worktime-am,time;worktime-pm
Block page¶
The block page is a CGI used to inform the user about the block reason. It’s a single page which can handle requests from ufdbGuard and SquidClamav (squidclamav-section).
The page is localized depending on browser language.
This configuration can be overwritten using RedirectUrl
property.
Troubleshooting¶
Useful test commands:
echo "http://bit.ly 10.10.0.1/ - - GET" | /usr/sbin/ufdbgclient -d
echo "http://bit.ly 10.10.0.1/ user@mydomain.com - GET" | /usr/sbin/ufdbgclient -d
Logfiles:
/var/log/ufdbguard/ufdbguardd.log
/var/log/ufdbguard/ufdbgclient.log
nethserver-sssd¶
This package implements authentication and user management layers. Supported authentication providers are:
- OpenLDAP with POSIX attributes
- Samba or Windows Active Directory
It includes the following parts:
- SSSD configuration
- events for users and groups management
- web interface for user management
- password policy management
- system validators for users and groups
- SSSD perl library to ease the implementation of e-smith templates
The implementation can work in two modes:
- read-and-write: if nethserver-dc or nethserver-directory are installed, the system will provide all user management features like creation, modification and deletion of users and groups
- read-only: if users and groups are read from a remote source, the system will be able to consume them only using passwd database
Configuration DB format¶
sssd=service
AdDns=192.168.1.12
BindDN=ldapservice@AD.EXAMPLE.COM
BindPassword=cjnsdkuyf8934tjhvsdkljcvsdv
LdapURI=
Provider=ad
Realm=AD.EXAMPLE.COM
Workgroup=EXAMPLE
DiscoverDcType=dns
ShellOverrideStatus=disabled
status=enabled
AdDns
: IP address of the Active Directory DNS serverLdapURI
: LDAP server URI (useldap://
orldaps://
scheme)DiscoverDcType {dns,ldapuri}
:dns
query SRV records in AD DNS to find the DC name;ldapuri
retrieve the DC name from theLdapURI
prop valueShellOverrideStatus
(defaultdisabled
) Ifenabled
, the value of the passwd user’s shell field- is set to
/bin/bash
and is no more read from the accounts provider. The eventnethserver-sssd-save
will expand and restart all services which use this property
Realm and workgroup¶
When the system is configured to use an Active Directory provider (Provider=ad
),
make sure to correctly set both Realm
and Workgroup
properties:
- Realm: this is the Kerberos realm and it’s case sensitive, but it’s usually configured in upper case as best practice. When the realm is used for DNS queries, it’s automatically forced to lower case.
- Workgroup: Samba NetBIOS name, maximum length is 15 characters. It’s usually the first part of the Realm in upper case
Events¶
Defined events are:
user-create¶
The event creates the user record inside the account provider database.
Parameters:
- username: must be unique
- name: full name of the user
- shell: default to /usr/libexec/openssh/sftp-server, if set to /bin/bash the user will be able to access the server using SSH
user-modify¶
The event changes the full name inside the account provider databases
Parameters:
- username
- name: full name of the user
- shell: default to /usr/libexec/openssh/sftp-server, if set to /bin/bash the user will be able to access the server using SSH
Note: shell option can’t be changed for AD users
user-cleanup¶
This event deletes all data (home directory) and the emails of the user (given as parameter) if the email server is installed. This event is bind to the event user-delete. Every application which needs to remove the data of a user, must use this event to trigger a specific action, made to remove the data.
Parameters:
- username
user-lock¶
The event locks the user preventing the access. All new users are in locked state.
Parameters:
- username
user-unlock¶
The event unlocks the user preventing the access. This event should be called after the invoking password-modify event for the user.
Parameters:
- username
group-create¶
The event creates the group record inside the account provider database.
Parameters:
- groupname: must be unique
- members: a list of users member of this group
group-modify¶
The event changes the members of a group inside the account provider database.
Parameters:
- groupname: must be unique
- members: a list of users member of this group
group-delete¶
This event deletes a group record from the the account provider database.
Parameters:
- groupname
password-policy-update¶
This event configures password expiration of a single user or of all users.
Parameters
username (optional)
passexpires: it can be yes or no. If user is set and value is yes, the user password will expires after a predefined number of days (see passwordstrength{MaxPassAge})
The duration of a password can be passwordstrength{MaxPassAge}
nethserver-sssd-remove-provider¶
This event removes any installed local account provider and also wipes the configuration of any remote account provider.
Before resetting the configuration, all accounts are dumped inside the following files in tsv format:
- /var/lib/nethserver/backup/users.tsv
- /var/lib/nethserver/backup/accounts.tsv
System users and groups¶
SSSD can access all users and groups from an account provider, but the Server Manager hides system users and groups.
The following users will not be accessible from the Server Manager:
- all users listed inside /etc/nethserver/system-users
- all users in /etc/passwd
The following groups will not be accessible from the web interface:
- all groups listed inside /etc/nethserver/system-groups
- all groups in /etc/group
The users and groups lists are retrieved by the following UI helpers:
/usr/libexec/nethserver/list-users
/usr/libexec/nethserver/list-groups
The number of entries returned by the server is limited. For instance, AD has a 1000 entries search results cap.
To retrieve the members of a group and the membership of a specific user:
/usr/libexec/nethserver/list-group-members
/usr/libexec/nethserver/list-user-membership
The Dashboard account counters are provided by:
/usr/libexec/nethserver/count-accounts
All those helpers support the -A
flag, to include hidden entries,
and the -s
flag to return entries without @domain
suffix.
AD LDAP search¶
The Samba net ads search -k
command can run an LDAP search against the AD
LDAP servers. The command requires a valid Kerberos ticket and a configured
environment variable, KRB5CCNAME
, pointing to it.
The krb5exec
command can set up the Kerberos ticket by authenticating with
the machine credentials, providing the same environment where the UI helpers
run, as explained in the previous section.
Putting the two commands together, the following command retrieves the admin
account record from AD LDAP.
krb5exec net ads search -k sAMAccountName=admin
The same command with ldapsearch
krb5exec ldapsearch -Y GSSAPI -b <BIND_PATH> -h <LDAP_SERVER_NAME> sAMAccountName=admin
Replace <BIND_PATH>
and <LDAP_SERVER_NAME>
with values provided by
net ads info
NethServer::SSSD¶
NethServer::SSSD is the Perl library module to retrieve current LDAP configuration. It supports both Active Directory and OpenLDAP providers.
Template example:
{
use NethServer::SSSD;
my $sssd = NethServer::SSSD->new();
$OUT .= "{ldap_uri, [".$sssd->ldapURI()."]}\n";
if ($sssd->isAD()) {
$OUT .= "{ldap_uids, [\"sAMAccountName\"]}.\n";
}
}
All functions are documented using perldoc
perldoc NethServer::SSSD
This command prints out the current settings, by querying NethServer::SSSD
methods. It requires the package openldap-clients
/usr/sbin/account-provider-test dump
Check the bind credentials are OK
/usr/sbin/account-provider-test
Join Active Directory¶
The Active Directory join operation is run by realmd. After the AD has been joined sucessfully the system keytab file is initialized as long as individual service keytabs, as defined on the respective service record (see `Service configuration hooks`_).
Leave and Re-Join Active Directory¶
To leave a remote AD go to the Accounts provider page.
For local AD provider, this is the manual leave procedure
realm leave
realm leave # two times
If the machine password or system keytab get corrupted, joining again the DC can fix them:
realm join -U admin $(config getprop sssd Realm)
…at prompt, type the admin’s password, then:
signal-event nethserver-sssd-save
If you leave and do not want to re-join, disable the sssd service permanently:
config setprop sssd status disabled Provider none
signal-event nethserver-sssd-save
signal-event nethserver-sssd-leave
signal-event nethserver-dnsmasq-save
Change the FQDN¶
Once we are bound to an account provider the FQDN cannot be changed any more. However, this procedure can be useful in early server configuration to fix a wrong FQDN. Please note that any existing account setting must be fixed manually. The procedure to do it is currently undefined.
For local account providers:
- Execute the leave procedure explained above
- Go to page System name and change the domain suffix in the FQDN field.
- Re-join as explained above
For remote account providers the procedure is similar. Use the Accounts provider page to leave/join the domain.
Service configuration hooks for Kerberos/GSSAPI authentication¶
A service (i.e. dovecot, squid) record in the configuration
DB can be extended with
the following special props, that are read by smbads
to create a Kerberos keytab file
for the service
dovecot=service
...
KrbStatus=enabled
KrbKeytabPath=/var/lib/dovecot/krb5.keytab
KrbPrimaryList=smtp,imap,pop
KrbKeytabOwner=
KrbKeytabPerms=
KrbStatus {enabled,disabled}
This is the main switch. If not set toenabled
,smbads
ignores the record when the service keytab is written.KrbKeytabPath
Keytab file path. If empty,/var/lib/misc/nsrv-<service>.keytab
is assumedKrbPrimaryList <comma separated words list>
Defines the keytab contents. In Kerberos jargon a “primary” is the first part of the principal string, before the slash (/
) character. Any primary in this list is exported to the keytab.KrbKeytabOwner
The unix file owner. Default is theservice
name. This is applied to both the credentials cache file and the keytab file.KrbKeytabPerms
The unix bit permissions in octal form. Default is0400
. This is applied to both the credentials cache file and the keytab file.
The implementation is provided by /usr/libexec/nethserver/smbads
.
Individual services can link themselves to nethserver-sssd-initkeytabs
action in the respective -update
event.
It is up to the KDC administrator to configure the SPN in the KDC. For instance, with Active Directory run the following commands to add the SPNs for Dovecot:
kinit admin@$(config getprop sssd Realm)
net ads setspn list
net ads setspn add $(hostname -s) imap/$(hostname)
net ads setspn add $(hostname -s) pop/$(hostname)
net ads setspn add $(hostname -s) smtp/$(hostname)
kdestroy
signal-event nethserver-sssd-save
Account import scripts¶
There are some perl scripts under the documentation scripts/
directory.
rpm -qd nethserver-sssd
import_users¶
It is possible to create user accounts from a TSV (Tab Separated Values) file with the following format:
username <TAB> fullName <TAB> password <NEWLINE>
Sample invocation:
import_users users.tsv
Alternative separator character:
import_users users.csv ','
import_groups¶
It is possible to create groups from a TSV (Tab Separated Values) file with the following format:
groupname <TAB> member1 <TAB> ... <TAB> memberN <NEWLINE>
Sample invocation:
import_users groups.tsv
Alternative separator character:
import_groups groups.csv ','
import_emails¶
It is possible to create mail aliases from a TSV (Tab Separated Values) file with the following format:
username <TAB> emailaddress <NEWLINE>
See import_users
section for a sample script invocation.
Password policy¶
The system can handle global or per-user policies. All policies are enforced by
PAM and saved under passwordstrength
inside the configuration
database.
Available properties are:
Users
: change strength password for all users, can be:strong
: (default) strong passwords must conform to cracklib checksnone
: no strength check
PassExpires
: can beyes
(default) orno
. If set tono
password will not expire, if set toyes
,following properties apply:
MaxPassAge
: minimum number of days for which the user is forced to keep the same password (default 0)MinPassAge
: maximum number of days for which the user can keep the same password (default: 180)PassWarning
: a shell warning is displayed to the user X days before password expiration
Configuration can be applied using the password-policy-update event.
DB example:
passwordstrength=configuration
MaxPassAge=180
MinPassAge=0
PassExpires=no
PassWarning=7
Users=none
nethserver-subscription¶
Manage NethServer Subscription clients
- Monitor the machine using Collectd and route notifications to a remote server
- Collect and send server informations to a remote server
- Download and install RPM updates automatically by configuring YUM repositories and nethserver-yum-cron
Database¶
Configuration is stored inside the configuration
database under the subscription
key.
Properties:
AlertsUrl
: URL used to send alerts and heartbeatAlertsAutoUpdates
: if set toenabled
, custom alerts will be downloaded each night fromAlertsAutoUpdatesUrl
.AlertsAutoUpdatesUrl
: URL used to fetch custom alerts databaseApiUrl
: URL called from the UI to retrieve subscription plan informationsInventoryUrl
: API endpoint where the data are sentNsRelease
: exact NethServer release request to mirrorlistSecret
: sysem secret key generated by Dartagnan (https://github.com/nethesis/dartagnan)PricingUrl
: URL used from the UI to access the subscription plansSystemId
: system unique idSiteUrl
: Product website URLDocsUrl
: Product documentation URLHelpdeskUrl
: Product helpdesk URLExtraRepositories
: Enable extra repositories in the subscription (comma separated : nethforge,repoA,repoB).
Example:
subscription=configuration
AlertsAutoUpdates=enabled
AlertsUrl=https://my.nethserver.com/api/
ApiUrl=http://my.nethserver.com/api/
InventoryUrl=http://my.nethserver.com/api/inventories/store/
NsRelease=7.4.1708
Secret=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
PricingUrl=https://my.nethserver.com/?action=newServer
SystemId=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
SiteUrl=https://www.nethserver.org/
DocsUrl=http://docs.nethserver.org/
HelpdeskUrl=https://community.nethserver.org/
Configuration¶
NethServer Subscriptions (by Nethesis) – run
config setprop subscription AlertsUrl https://my.nethserver.com/api/machine/ config delprop subscription AlertsAutoUpdatesUrl config setprop subscription InventoryUrl https://my.nethserver.com/api/machine/inventories/store/ signal-event nethserver-subscription-update
NethServer Enterprise partner program (by Nethesis) – run
config setprop subscription AlertsUrl https://my.nethesis.it/isa/ config setprop subscription AlertsAutoUpdatesUrl https://my.nethesis.it/api/ config setprop subscription InventoryUrl https://my.nethesis.it/isa/inventory/store/ signal-event nethserver-subscription-update
YUM repositories¶
Once NethServer Subscriptions is activated, YUM configuration is changed to contact the subscribed repositories. Every night the list of enabled repositores will be reset to meet the subscription requirements.
The /etc/e-smith/events/actions/nethserver-subscription-eorepo
action configures YUM
repositories based on subscription.
The behavior of the script can be changed using /etc/nethserver/eorepo.conf
file which
may contain the list of repositories to be enabled.
Restore original NethServer repositories¶
To revert the system configuration to its initial state and use the original NethServer repositories run
config setprop subscription SystemId '' Secret ''
signal-event software-repos-save
Enable extra repositories in subscription¶
If you need to enable an extra repository like NethForge, you can do it by the command line
config setprop subscription ExtraRepositories nethforge
signal-event software-repos-save
The list of repositories (comma separated : nethforge,repoA,repoB) must match a valid repository definition list in /etc/yum.repos.d. Beware it is case sensitive.
Events¶
nethserver-subscription-update
standard update event; it runs on new installation and subsequent updatesnethserver-subscription-save
runs when the registration token is set from the UI, altering the YUM repository configurationrestore-yumrepos
runs duringpre-restore-config
event. Actions and templates bound to it run in a temporary environment where theconfiguration
DB is extracted from the configuration backup and temporarily applied to the system. It occurs before automatic installation of packages from the backup set. It is the point where the YUM configuration is restored to make subscribed repositories immediately available.
Alerts¶
Collectd listens for notification on a well known socket: /var/run/collectd.sock
.
Notifications can be generated by Collectd thresholds, cron jobs or even external daemons like NMS.
Collectd thresholds¶
You can trigger an alarm named testalarm
from command line:
echo -e "PUTNOTIF host=$(hostname) type=custom type_instance=testalarm severity=failure time=$(date +%s) message=\"$1 FAILURE\"" | nc -U /var/run/collectd.sock &>/dev/null
You can also resolve the above alarm:
echo -e "PUTNOTIF host=$(hostname) type=custom type_instance=testalarm severity=okay time=$(date +%s) message=\"$1 OK\"" | nc -U /var/run/collectd.sock &>/dev/null
Adding a threshold configuration¶
Create a /etc/collectd.d/threshold.conf
template fragment using collectd threshold syntax. Enjoy collectd docs here https://collectd.org/documentation/manpages/collectd-threshold.5.shtml
Collectd python plugin¶
The nethserver_alerts.py file implements a Collectd plugin. The plugin dispatches each notification to remote Dartagnan instance. If the remote Dartagnan instance is not available, the plugin will save the notifications inside a local queue and will retry to resend them later.
Plugin options (inside /etc/collectd.d/threshold.conf
):
api_url
URL where alerts will be dispatchedlk
: System IDsecret
: (optional) if set, the secret will be used as authorization tokendebug
: enable the debug, can beyes
orno
. Default tono
NethServer Monitor Services (NMS)¶
NMS is a python daemon which monitor the status of all services registered inside the configuration db. Each service is checked every 300 seconds, if the status of a service changes, NMS writes a notification to Collectd socket.
Inventory¶
The inventory uses the facter
software, from puppet
(https://docs.puppet.com/facter/). Facter collects a standard set of
informations (like CPU model, runing kernel, etc.) but the system also gather
some custom facts
.
Custom facts¶
Scripts for custom facts are inside
/opt/puppetlabs/puppet/lib/ruby/2.1.0/facter/
directory. Each package can
register a new custom fact, but this packages always provides the following:
rpms
, list of installed RPMsraid
, RAID configurationtemplates_custom
, list of configured template custom wihout contentarp_macs
, number of network devicesusers
, list of configured usersbackup
, info about backup statusesmithdb
, all esmith databases, password values are replaced with***
Sending the inventory¶
Make sure that SystemId
property under subscription
key is already set.
If the Secret
property is set, it will be used as authorization token when
sending the inventory.
The inventory will be sent every day using cron. If you want to manually send the inventory, use the following command:
/etc/cron.daily/nethserver-inventory
nethserver-suricata¶
The IPS (Intrusion Prevention System) module configures Suricata using the netfilter queue (NFQUEUE). NFQUEUE is an iptables and ip6tables target which delegate the decision on packets to a userspace software.
All traffic will be analyzed by Suricata itself and events are logged inside /var/log/suricata/eve.json
.
See EveBox for a report of blocking and alerting rules.
Suricata rules are managed by Pulledpork.
Manually enable/disable Suricata¶
Enabling:
config setprop suricata status enabled
signal-event firewall-adjust
signal-event nethserver-suricata-save
Disabling:
config setprop suricata status disabled
signal-event firewall-adjust
signal-event nethserver-suricata-save
Bypass¶
All bypasses are saved inside the ips
database.
Each record with bypass
type has the following properties:
Host
: it can be a firewall object or a raw IP/CIDR addressstatus
: it can beenabled
ordisabled
Description
: optional description
Example:
bypass1=bypass
Description=
Host=host;test1
status=enabled
bypass2=bypass
Description=
Host=192.168.0.1
status=disabled
Custom rules¶
If a file named /etc/suricata/rules/custom.rules
, it will be included inside Suricata configuration.
After creating the file, execute: signal-event nethserver-suricata-update
.
Also remember to add it to the configuration backup:
echo /etc/suricata/rules/custom.rules >> /etc/backup-config.d/custom.include
Troubleshooting¶
When troubleshooting network traffic, just remember that Suricata will intercept all the traffic.
nethserver-unbonund¶
Unbound is a validating, recursive, and caching DNS resolver. Currently it’s only used only as DNS server for anti-spam DNSBL implementation.
When a new mail arrives, the anti-spam filter checks if sender IP address is listed in well-known blacklists. Some blacklists doesn’t allow multiple query from public DNS servers, thus the system needs a DNS server which can directly query the blacklist DNS.
If the mail filter is installed, dnsmasq configuration is changed, and specific
domain query are redirect trough unbond.
See: /etc/e-smith/templates/etc/dnsmasq.conf/26unbound_rbl
.
Package: nethserver-unbound.
Database¶
Configuration is saved in unbound
key inside configuration
database.
Available properties:
UDDPort
: server address of the mail server, default islocalhost
access
: default isnone
since unbound should not be used as a normal DNS serverstatus
: can beenabled
ordisabled
, default isenabled
Example:
unbound=service
UDPPort=10053
access=none
status=enabled
Configuration can be applied using the nethserver-unbound-update
event.
nethserver-unixODBC¶
This package contains templates that generate ODBC configuration.
/etc/odbc.ini¶
The template that generates this file scan all configuration db searching for a key that has type=odbc (or ODBC for backward compatibility with SMEserver) and generates a section in /etc/odbc.ini. For example, this is ODBC object is used for asterisk cdr database:
# config show odbc-asteriskcdr
odbc-asteriskcdr=odbc
Database=asteriskcdrdb
Description=ODBC on asteriskcdrdb
Driver=MySQL
Port=3306
Server=localhost
this object generates the /etc/odbc.ini section:
[odbc-asteriskcdr]
Server = localhost
Database = asteriskcdrdb
Driver = MySQL
Port = 3306
Description = ODBC on asteriskcdrdb
/etc/odbcinst.ini¶
This templates contains configuration for ODBC drivers. By default there are only MySQL and PostgreSQL driver configurated. The nethserver-unixODBC package requires those to be installed to:
# rpm -q --requires nethserver-unixODBC
...
mysql-connector-odbc
postgresql-odbc
...
Usage¶
Creation of a new ODBC driver is as simply as launching:
config set <new ODBC object name> odbc Description <description> Driver <MySQL|PostgreSQL> Server <server hostname> Database <database name> Port <database port>
signal-event nethserver-unixODBC-update
nethserver-vsftpd¶
The FTP module uses the vsftpd daemon. It is accessible only from local network and it is disabled by default.
It supports both virtual and system users, but not at the same time.
All virtual home directories are created inside /var/lib/nethserver/ftp
.
The daemon will run as ftp
user with passive mode enabled.
Virtual users¶
Virtual users are enabled by default.
Add a new user¶
Create a record inside the accounts db and activate changes:
db accounts set <user> ftp status enabled Password <pass> Chroot enabled
signal-event nethserver-vsftpd-save
The event will generate /etc/vsftpd/ftpusers.db
(Berkeley db) containing all user credentials.
The db is used for pam authentication (see: /etc/pam.d/vsftpd-virtual
).
Properties:
status
: can beenabled
ordisabled
. Ifenabled
, the user can access the ftp serverPassword
: the user password in clear text, can not be blankChroot
: can beenabled
ordisabled
. Ifenabled
, the user will be chrooted to/var/lib/nethserver/ftp/<user>
directory. Ifdisabled
the user is not chrooted. ATTENTION: non-chrooted users can explore the entire file systemChrootDir
: set a custom ChrootDir, it may be used to chroot a user inside a shared folder
System users¶
Warning
This configuration is highly discouraged, because user’s password is transmitted in clear text
After enabling system users, all virtual users will be disabled.
Enable system users:
config setprop vsftpd UserType system
signal-event nethserver-vsftpd-save
Enable an existing system user to access FTP server:
db accounts setprop myuser FTPAccess enabled
db accounts setprop myuser Shell /bin/bash
signal-event user-modify myuser
Apply configuration:
signal-event nethserver-vsftpd-save
To disable an already enabled user:
db accounts setprop myuser FTPAccess disabled
signal-event nethserver-vsftpd-save
If not explicitly disabled, all system users are chrooted inside their home directories. To disable a chroot for a system user:
db accounts setprop myuser FTPChroot disabled
signal-event nethserver-vsftpd-save
Custom chroot¶
When the FTP server uses system users, custom chroot is not supported: all users are chrooted inside their own home directory. Although it’s possible to change a system user home directory. Use the command below if the system user will used only for FTP access:
lusermod -d <your_custom_home> <user>
nethserver-webtop4¶
WebTop 4 is a full-featured groupware written in Java.
It’s composed by three parts:
- Java web application running on Tomcat 7
- PHP implementation of Active Sync protocol
- PostgreSQL database
Access to web application is forced in SSL mode.
Database¶
Configuration is saved in webtop
key inside configuration
database.
Available properties:
PublicUrl
: public URL used to publish resources for the cloud. If not set, default ishttp://<FQDN>/webtop
ActiveSync
: if set toenabled
, it enables /Microsoft-Server-ActiveSync url. Default isenabled
Example:
webtop=configuration
ActiveSync=enabled
PublicUrl=
Configuration can be applied using the nethserver-webtop4-update
event.
Troubleshooting¶
In case of errors, see following logs:
- Tomcat:
/var/log/tomcat/catalina.out
- Active Sync:
/var/log/z-push/z-push-error.log
To inspect z-push status use:
php /usr/share/webtop/z-push/z-push-admin.php
Known problems¶
When PostgreSQL is restarted, WebTop can loss the database connection. If a blank page is displayed, restart Tomcat with the following command:
systemctl restart tomcat
nethserver-webtop5¶
WebTop 5 is a full-featured groupware written in Java.
It’s composed by three parts:
- Java web application running on Tomcat 8
- PHP implementation of Active Sync protocol
- PHP implementation of CardDAV and CalDAV protocol
- PostgreSQL database
Access to web application is forced in SSL mode.
WebTop 5 has been split in 4 different RPMs:
- webtop5: Tomcat webapp and all third-party jars, derived from a WAR. It contains all jars developed by Sonicle. This package will be updated at each WebTop release
- webtop5-zpush: ActiveSync implementation for WebTop, it contains PHP code from z-push project (http://z-push.org/)
- webtop5-webdav: CardDAV and CalDAV implementation for WebTop, it contains PHP code from sabre/dav project (http://sabre.io/dav/)
- nethserver-webtop5: NethServer auto-configuration for WebTop
Database¶
Configuration is saved in webtop
key inside configuration
database.
Available properties:
ActiveSyncLog
: log level of z-push implementation. As default z-push will log only relevant errors.ActiveSyncLegactIds
: can beenabled
ordisabled
. If set toenabled
, use backward compatibile z-push ids to avoid device full rsync on update. See “Active Sync” section for more infoDebug
: if set totrue
, enable debug for the web application. Default isfalse
DefaultLocale
: default locale for WebTop users. To list available locales execute:/etc/e-smith/events/actions/nethserver-webtop5-locale-tz
DefaultTimezone
: default timezone for WebTop users. To list available timezones:JAVA_HOME=/usr/share/webtop/ java ListTimeZones
MinMemory
andMaxMemory
: minimun and maximum memory of Tomcat instance. Values are expressed in MB.PublicUrl
: public URL used to publish resources for the cloud. If not set, default ishttp://<FQDN>/webtop
DavServerUrl
: Dav server URL for CalDAV and CardDAV clients configuration. If not set, default ishttps://<FQDN>/webtop-dav/server.php
DavServerLog
: log level of webtop-dav implementation. As default webtop-dav will log only relevant errors.PbxProvider
: PBX provider namePbxProviderNethvoiceWebrestUrl
: NethVoice base url for API calls, used whenPbxProvider
is set tonethvoice
DefaultToolBarIconsSize
: Default dimension of the toolbar icons, available values:small
,medium
,large
. Default ismedium
RemoteCalendarAutosync
: if set toenabled
, it enables remote calendars auto-sync functionality. The sincronization interval can be set by user on remote calendar creation. Default isenabled
RemoteCalendarAutosyncOnlywhenonline
: if set toenabled
the remote calendars is auto-sync only when calendar’s owner is online during the sincronization time. Default isdisabled
RemoteCategoryAutosync
: if set toenabled
, it enables remote categories auto-sync functionality. The sincronization interval can be set by user on remote category creation. Default isenabled
RemoteCategoryAutosyncOnlywhenonline
: if set toenabled
the remote categoties is auto-sync only when category’s owner is online during the sincronization time. Default isdisabled
VirtualHost
: set custom virtual host, e.g. mygroupware.mydomain.it. Virtual host can be used to access WebTop and configure ActiveSync/CalDAV/CardDAV clientsKnownDeviceVerification
: if set toenabled
, a notification will be sent for any new authentication attempt from an unknown-device, defaultdisabled
KnownDeviceVerificationRecipients
: a comma-separated list of email addresses to use as CCN recipients that will receive a copy of the unknown-device noticeKnownDeviceVerificationNetWhitelist
: a comma-separated list of networks, specified in CIDR format, from where all devices will be considered trusted, and no notifications will be sent
Example:
webtop=configuration
ActiveSyncLog=ERROR
Debug=false
DefaultLocale=en_US
DefaultTimezone=Etc/UTC
MaxMemory=1024
MinMemory=512
PublicUrl=
Configuration can be applied using the nethserver-webtop5-update
event.
Reset admin password¶
Access the database
su - postgres -c 'psql webtop5'
Copy & paste the following query:
UPDATE "core"."local_vault" SET "password_type"='PLAIN', "password"='admin' WHERE ("domain_id"='*') AND ("user_id"='admin');
Access the web interface using
admin
user with passwordadmin
.
Troubleshooting¶
Please note that nethserver-webtop5 is composed by many parts. Each part has its own logs and troubleshooting best practices.
Web application¶
The web application logs are inside /var/log/webtop/webtop.log
.
Tomcat¶
Tomcat instance is managed by systemd unit called tomcat8@webtop
.
All logs are saved inside /var/lib/tomcats/webtop/logs/
directory.
The logs are rotated daily and deletes after 2 days.
Active Sync¶
Active Sync is implemented using a PHP application called z-push.
All logs are inside /var/log/z-push/
directory.
To inspect z-push status use:
sudo -u apache scl enable rh-php73 'php /usr/share/webtop/z-push/z-push-admin.php'
It is also possibile to enable z-push debug using these commands:
config setprop webtop ActiveSyncLog DEBUG
signal-event nethserver-webtop5-update
Instead of DEBUG
you can use any constant supported by z-push implementation,
but remove the LOGLEVEL_
prefi.
See /usr/share/webtop/z-push/inc/zpush.config.php
.
You can test Active Sync using this command (please set user, password and server_name):
curl -k -u goofy@local.neth.eu:password https://server_name/Microsoft-Server-ActiveSync
You should see an HTML output containing the string:
GET not supported
Legacy ids¶
When the ActiveSyncLegacyIds
is set to enabled
, the z-push implementation is affected by the following limitations:
- a user can’t have more than one calendar with the same name
- resources with very long names (eg. calendars) can cause synchronization problems
If such problems occur, please switch to new id implementation:
config setprop webtop ActiveSyncLegacyIds disabled
rm -rf /var/log/z-push/state/*
signal-event nethserver-webtop5-update
Please note that after switching to new implementation, all devices will require a full synchronization.
CardDAV and CalDAV¶
CardDAV and CalDAV are implemented using a PHP application called webtop-dav.
All logs are inside /var/log/webtop-dav/
directory.
It is also possibile to enable webtop-dav debug using these commands:
config setprop webtop DavServerLog DEBUG
signal-event nethserver-webtop5-update
Instead of DEBUG
you can use any constant supported by webtop-dav
implementation.
See /usr/share/webtop/webtop-dav/lib/webtop/Log.php
.
To enable browser-plugin
for directory indexes of the Dav server:
config setprop webtop Debug true
signal-event nethserver-webtop5-update
Tomcat instance¶
WebTop uses its own Tomcat instance running on port 58080
.
The instance is launched with some special Java options,
see content of /etc/sysconfig/tomcat8@webtop
.
License Headers¶
Always add a COPYING
in the documentation directory of your
package, containing the full GPLv3 license.
Change the Copyright line according to your needs.
PHP, Javascript, CSS¶
/*
* Copyright (C) 2020 Nethesis S.r.l.
* http://www.nethesis.it - nethserver@nethesis.it
*
* This script is part of NethServer.
*
* NethServer is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License,
* or any later version.
*
* NethServer is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with NethServer. If not, see COPYING.
*/
Perl, Python and BASH¶
#
# Copyright (C) 2020 Nethesis S.r.l.
# http://www.nethesis.it - nethserver@nethesis.it
#
# This script is part of NethServer.
#
# NethServer is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License,
# or any later version.
#
# NethServer is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with NethServer. If not, see COPYING.
#
Rebranding Administrator Manual¶
It’s possible to create a custom version of the Administrator Manual.
Environment¶
Clone the documentation repository
Create a directory inside the main
administrator-manual/<lang>
directory. Example for a new rebranding called NethService and available only in Italian:mkdir administrator-manual/it/nethservice
Enter the directory, and create the structure:
cd nethservice mkdir _templates _static _build _themes
Copy the makefile and configuration from parent directory:
cp ../Makefile ../conf.py .
Contents¶
First, create a custom index.rst
with required chapters. Example:
My section
-------------
.. toctree::
:maxdepth: 2
installation
newchapter
To add a chapter, create new rst file inside the current directory. Example for newchapter.rst
:
===========
New chapter
===========
This is a new chapter.
If you wish to reuse existing chapters, create links to the parent directory. Example:
ln -s ../installation.rst installation.rst
Product name and version¶
Edit the conf.py
by setting product name and version. Feel free to customize anything you need but make sure to edit at least following variables:
- project
- release
Create a rst_prolog
file with the macro for product name and download site. Content of rst_prolog
file:
.. |product| replace:: NethService
.. |download_site| replace:: http://www.nethesis.it
Theme¶
Choose and existing Sphinx theme or copy a new theme inside the _themes
directory.
If you want to use a custom theme, remember to set following variables inside conf.py
:
html_theme_path = ['_themes']
html_theme = 'mynewshinytheme'
Artworks¶
If you wish to add custom artworks like logo and favicon, edit these variables inside conf.py
:
html_static_path = ['_static']
html_logo = '_static/logo.png'
html_favicon = '_static/favicon.ico
License¶
This documentation is distributed under the terms of Creative Commons - Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license.

You are free to:
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material
The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- NonCommercial — You may not use the material for commercial purposes.
- ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
This is a human-readable summary of (and not a substitute for) the full license available at: http://creativecommons.org/licenses/by-nc-sa/4.0/
Architecture documentation is from SME Server project and is licensed under GNU Free Documentation License 1.3 (http://www.gnu.org/copyleft/fdl.html). See http://wiki.contribs.org/ for original documentation.