User Tools

Site Tools


doc:appunti:linux:sa:postfix_spamassassin_clamav_dovecot

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
doc:appunti:linux:sa:postfix_spamassassin_clamav_dovecot [2021/03/01 09:53] – [Quota] niccolodoc:appunti:linux:sa:postfix_spamassassin_clamav_dovecot [2023/10/30 11:09] (current) – [Enable SSL, imaps and pop3s] niccolo
Line 64: Line 64:
  
 <file> <file>
-!include auth-system.conf.ext+# Order matters: we use passwdfile first because it is most likely used 
 +# and because system autentication (PAM) does a 2-seconds delay on fail.
 !include auth-passwdfile.conf.ext !include auth-passwdfile.conf.ext
 +!include auth-system.conf.ext
 </file> </file>
  
Line 174: Line 176:
  
 Once restarted the **dovecot.service**, we will find the services listening on the well known ports, and we can check the SSL certificates. Once restarted the **dovecot.service**, we will find the services listening on the well known ports, and we can check the SSL certificates.
 +
 +==== Client connections are being dropped ====
 +
 +You may encounter wanring messages like these into the mail log:
 +
 +<code>
 +dovecot: master: Warning: service(imap-login): process_limit (100) reached, client connections are being dropped
 +</code>
 +
 +To increase the number of process that are allowed to spawn, edit the **/etc/dovecot/conf.d/10-master.conf** file and set
 +
 +<file>
 +default_process_limit = 200
 +</file>
 +
 ===== SpamAssassin Filter ===== ===== SpamAssassin Filter =====
  
Line 207: Line 224:
   * **clamdscan**   * **clamdscan**
   * **clamav-freshclam**   * **clamav-freshclam**
 +  * **libclamunrar9** (non-free package to scan inside RAR archives)
  
 Periodic download (update) of viruses database is performed by the **clamav-freshclam.service**, you can check the **/var/log/clamav/freshclam.log**. Periodic download (update) of viruses database is performed by the **clamav-freshclam.service**, you can check the **/var/log/clamav/freshclam.log**.
Line 249: Line 267:
 mailbox_command = /usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT" mailbox_command = /usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"
 </file> </file>
 +
 +==== Error stats-writer Broken pipe ====
 +
 +**WARNING**: At least in **Debian 11 Bullseye** it seems that the **dovecot-lda** sometimes (not always!) tries to write to the **Dovecot Stats Service** using the **/run/dovecot/stats-writer** socket. Unfortunately the socket is owned by **root:dovecot** and has mode **0660** (default Debian settings), whereas the LDA is invoked by Postfix with just the user's privileges. So you will find error messages like this into the mail.log:
 +
 +<code>
 +postfix/local[76866]: 7E67B7D3F4: to=<user@domain.tld>, ...,
 +    status=bounced (Command died with status 134:
 +    "/usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"".
 +    Command output: lda(lica-marilena):
 +    Error: net_connect_unix(/run/dovecot/stats-writer) failed:
 +    Permission denied Aborted Unable to flush stdout: Broken pipe )
 +</code>
 +
 +It seems that the quickest and simplest solution is to make the socket **0666 mode** (which will risk the stats service to be abused). Add the following snippet to **/etc/dovecot/conf.d/15-lda.conf**:
 +
 +<file>
 +service stats {
 +  unix_listener stats-writer {
 +    mode = 0666
 +  }
 +}
 +</file>
 +
 +See the following posts about the problem:
 +
 +  * [[https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=961033|stats-writer permission issue when using LDA via procmail - Debian bug #961033]]
 +  * [[https://dovecot.org/pipermail/dovecot/2022-February/124267.html|postfix, dovecot-lda, /run/dovecot/stats-writer socket permision and local user delivery, again]]
 +
 +==== Error Command output: Aborted ====
 +
 +Another subtle error with Dovecot LDA can be reported generically into the Postfix log as:
 +
 +<code>
 +postfix/local[1615363]: E85037D1C6: to=<user@domain.tld>, ...
 +    status=bounced (Command died with status 134:
 +    "/usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"".
 +    Command output: Aborted )
 +</code>
 +
 +To get some hints about the problem you can enable logging in Dovecot LDA; you can add the following in **/etc/dovecot/conf.d/15-lda.conf**:
 +
 +<file>
 +protocol lda {
 +  ...
 +  log_path = /var/log/dovecot-lda-errors.log
 +  info_log_path = /var/log/dovecot-lda.log
 +}
 +</file>
 +
 +and create the log files with suitable permissions (in my case the LDA is executed with user's privileges, so I made it world-writable).
 +
 +<code>
 +lda(username)<1619235><QdsSDJpj72MjtRgACo+hHQ>: Error: program
 +    exec:/usr/local/lib/dovecot/sieve-filter/spamc-filter.sh (1619240):
 +    Execution timed out (> 10000 msecs)
 +lda(username)<1619235><QdsSDJpj72MjtRgACo+hHQ>: Error: program
 +    exec:/usr/local/lib/dovecot/sieve-filter/spamc-filter.sh (1619240):
 +    Forcibly terminated with signal 15
 +lda(username)<1619235><QdsSDJpj72MjtRgACo+hHQ>: Panic:
 +    output stream (temp iostream in /tmp/dovecot.lda.
 +    for (program client seekable output)) is missing error handling
 +lda(username)<1619235><QdsSDJpj72MjtRgACo+hHQ>: Error:
 +    Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42)...
 +</code>
 +
 +There was a problem with **the filter program responding too slowly**. It is possible to change the **exec timeout**, in this case for the **filter** extension, just add the following in **/etc/dovecot/conf.d/90-sieve-extprograms.conf** (notice that you can configure ''sieve_pipe_exec_timeout'', ''sieve_filter_exec_timeout'' and ''sieve_execute_exec_timeout''):
 +
 +<file>
 +plugin {
 +  ...
 +  # Change the default timeout (10 seconds) for the filter extension.
 +  sieve_filter_exec_timeout = 60s
 +
 +  # Change the default timeout (10 seconds) for the execute extension.
 +  sieve_execute_exec_timeout = 60s
 +}
 +</file>
 +
 +==== Error Unable to flush stdout ====
 +
 +FIXME This error message has an unwknown origin.
 +
 +<code>
 +postfix/local[1485291]: 5BDBD7D1C6: to=<user@domain.tld>, ...
 +    status=bounced (Command died with status 134:
 +    "/usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"".
 +    Command output: Aborted Unable to flush stdout: Broken pipe )
 +</code>
  
 ===== Sieve filtering ===== ===== Sieve filtering =====
Line 309: Line 416:
 <code bash> <code bash>
 #!/bin/sh #!/bin/sh
-/usr/bin/spamc+/usr/bin/spamc --connect-timeout=5 --timeout=15
 </code> </code>
 +
 +Notice that we set two timeouts on spamassassin client, one to connect to the daemon and one to get the result. If the daemon does not respond timely, the message is passed unchanged. Beware that **the timeouts should be lower** than the one specified into the **Dovecot LDA Sieve Extprograms plugin** configuration.
  
 ==== How to Test an User's Sieve Filter File ==== ==== How to Test an User's Sieve Filter File ====
Line 358: Line 467:
     terminated with non-zero exit code 127     terminated with non-zero exit code 127
 </code> </code>
 +
 +FIXME How to handle errors in filter external commands? The default action is to send a **non-delivery notification**, but you may want to fall-back to deliver the message anyway (e.g. if your antispam filter goes broken).
  
 ==== Multiple Sieve Scripts ==== ==== Multiple Sieve Scripts ====
Line 401: Line 512:
  
 The files (with the .sieve extension) will be searched into the proper directory. The files (with the .sieve extension) will be searched into the proper directory.
 +
 +==== Using Sieve to decode winmail.dat attachments ====
 +
 +Microsoft Outlook uses the infamous **winmail.dat** attachment to forward emails: it is the proprietary **TNEF** format. In this article you can find a recipe to filter that attachments using a Sieve filter: **[[sieve_filtering_tnef]]**.
 +
  
 ===== Roundcube with MySQL ===== ===== Roundcube with MySQL =====
Line 769: Line 885:
 </code> </code>
  
 +**NOTICE**: The Dovecot's quota limit is the **soft limit** imposed by the filesystem.
  
 ==== Dovecot quota status service for Postfix ==== ==== Dovecot quota status service for Postfix ====
Line 795: Line 912:
 </file> </file>
  
-The Postfix quota-status service may uses also a **quota_over_flag** provided by the **userdb** backend. We don't have such a field into the userdb, neverthless we have to configure the parameters **quota_over_flag_value** and **quota_over_script**, otherwise the quota check is skipped entirely:+The Postfix quota-status service may uses also a **quota_over_flag** provided by the **userdb** backend. We don't have such a field into the userdb, neverthless we have to configure the parameters **quota_over_flag_value** and **quota_over_script**, otherwise the quota check is skipped entirely (don't be fooled if the ''doveadm'' tool is working, verify also that the Postfix check is actually working!):
  
 <file> <file>
 +# === WARNING ===
 +# Both "quota_over_flag_value" and "quota_over_script" are 
 +# required, otherwise the Postfix quota-status check will always 
 +# return DUNNO (i.e. user is under quota). This is because 
 +# without those two parameters the quota check is skipped 
 +# completely, as seen in the mail_debug = yes log:
 +# Debug: quota: quota_over_flag check: quota_over_script unset - skipping
 +#
 +# Actually we don't have a "quota_over_flag" field in userdb; 
 +# when quota-status service is called by Postfix, the value 
 +# quota_over_flag=0(*dummy*) is assumed due the config values 
 +# below. Current quota status for the user is then checked by 
 +# querying the filesystem and it will be 1 for overquota, 0 
 +# otherwise.
 +#
 +# As a side effect, the quota-warning script is executed at 
 +# every check if the user is overquota because 0 mismatches 1.
 plugin { plugin {
   quota_over_flag_value = FALSE   quota_over_flag_value = FALSE
Line 805: Line 939:
 </file> </file>
  
-We have to define also a **quota-warning service**, which basically is a script called when the user crosses some quota barriers, we can use it to send warning messages. The script needs to be run as root, because we want it to be able to switch to the final user. The **/var/run/dovecot/quota-warning** socket must be world-writable so the user can write to it.+We have to define also a **quota-warning service**, which basically is a script called when the user crosses some quota barriers, we can use it to send warning messages. The script needs to be run as root, because we want it to be able to switch to the final user (this is required by our system which uses Maildir and user's filters). The **/var/run/dovecot/quota-warning** socket must be world-writable so the user can write to it.
  
 <file> <file>
Line 828: Line 962:
 </file> </file>
  
-FIXME put here the script!+So, all after all, here it is the script executed when an user is crossing some quota levels and when the fake **quota_over_flag** does not match the actual quota status (i.e. it is called when the user is over quota, and we just do nothing in that case!): 
 + 
 +**/usr/local/bin/quota-warning.sh** 
 + 
 +<code bash> 
 +#!/bin/sh 
 +ARG1=$1 
 +USER=$2 
 +DATE="$(date -R)" 
 + 
 +# ==== WARGNING ==== 
 +#  * The user is set by Dovecot "service quota-warning", in our 
 +#    configuration is: uid=0(root) gid=0(root) groups=0(root) 
 +#  * Working directory is /run/dovecot/ 
 +#  * /tmp directory is /tmp/systemd-private-<...>-dovecot.service-<...>/tmp 
 + 
 +# Do nothing if called on Dovecot quota_over_flag_value mismatch. 
 +test "$ARG1" = "mismatch" && exit 0 
 + 
 +# Send the warning message. 
 +#cat << EOF | /usr/lib/dovecot/dovecot-lda -d $USER -o "plugin/quota=fs:User quota:noenforcing" 
 +cat << EOF | /usr/lib/dovecot/dovecot-lda -d $USER 
 +From: postmaster@supermail.texnet.it 
 +Subject: Attenzione: mailbox quasi piena 
 +Date: $DATE 
 +Content-Type: text/plain; charset=UTF-8; format=flowed 
 +Content-Transfer-Encoding: 8bit 
 + 
 +La tua mailbox è quasi piena; lo spazio occupato ha superato il ${ARG1}%. 
 +Eliminare i messaggi non più necessari ed eventualmente svuotare il cestino. 
 +EOF 
 +</code>
  
 ==== Troubleshoting Postfix quota-status service ==== ==== Troubleshoting Postfix quota-status service ====
Line 842: Line 1007:
 action=554 5.2.2 Quota exceeded (mailbox for user is full) action=554 5.2.2 Quota exceeded (mailbox for user is full)
 </file> </file>
 +
 +The **554 Quota exceeded error** should be returned when the **actual user's quota** plus the announced **size** will exceed the filesystem **soft quota** plus the **quota_grace** percentage.
 +
 +Another way to check the service using a single command line is using **netcat**:
 +
 +<code>
 +printf "recipient=user@domain.org\nsize=3000000\n\n" | nc localhost 12340
 +</code>
  
 In Postfix configuration **/etc/postfix/main.cf** add a restriction on smtpd (the order of restrictions matter): In Postfix configuration **/etc/postfix/main.cf** add a restriction on smtpd (the order of restrictions matter):
doc/appunti/linux/sa/postfix_spamassassin_clamav_dovecot.1614588817.txt.gz · Last modified: 2021/03/01 09:53 by niccolo