- Mailman-users - lists.mailman3.org

Question about "envelope-from" header
by Stephen Daniel Sept. 13, 2023

Sept. 13, 2023

Mailman Gurus -- I run a small mailman3 implementation in a VM inside Google's cloud. The same VM runs both mailman3 and postfix. Because outbound SMTP is blocked by Google, my postfix relays outbound mail through smtp-relay.gmail.com, using credentials from my google workspace account. Works pretty well. For reasons unrelated to my mailing lists, I'd like to have the same VM provide a couple of mail aliases. Mail that comes in to " alias1(a)mydomain.com" should be relayed to "random1(a)somedomain.com". I am trying to do this by setting up virtual aliases in postfix, as per http://www.postfix.org/virtual.5.html. This works except that smtp-relay.gmail.com rejects the forwarded message because of "invalid credentials". I believe that email going out through smtp-relay.gmail.com can only come from a small set of previously authorized domains, and a random person who sends to "alias1(a)mydomain.com" is unlikely to be on that list. I think I could fix this by having postfix edit the "envelope-from" header as it relays the mail, so that the "envelope-from" header contains something like "admin(a)mydomain.com". Not sure, but I think so. My question is this: If I edit the envelope-from header in my outbound emails to say "admin(a)mydomain.com" am I going to break my email lists? Will bounce processing still work? Will replies to the list still work? Thanks! -- Stephen Daniel

3 2

Mailman messages are getting a TAG in some email clients.....how????
by bob B Sept. 13, 2023

Sept. 13, 2023

So I know about the setting "Anonymous list" but regardless of that setting some email clients put the list name in [] when it displays messages in your inbox. Does anyone know how email clients are doing this? I have users who email clients do not do this, and was wondering if it is a setting in their client or just not possible. Here is a link to an image showing the tag for a list called TEST6 https://imgur.com/a/7VyJtsD

3 3

How to let mailman log every single email that it sends out?
by Mohsen Masoudfar Sept. 10, 2023

Sept. 10, 2023

I'm curious about the configuration required to have Mailman record logs for every individual email it sends. I see that it sends to the entire list, but I'm interested in obtaining log entries for each email sent to every individual subscriber on the list. How can I access this information? Is it a postfix setting, or a Mailman3 setting? I have checked /var/log/mail.log and all the logs under /var/log/Mailman3/* Mailman is running on Ubuntu 18.3 Thank you! Mohsen

4 5

Issue with Docker Postfix and Mailman3 Configuration
by recida8902＠picvw.com Sept. 9, 2023

Sept. 9, 2023

Dear Support Team, I am writing to seek assistance with an issue I am facing while configuring Docker Postfix with Docker Mailman3. I have successfully set up the configuration, and I can send emails without any problems. However, I am encountering an issue where incoming emails are not being received by the Mailman application. When I send an email from my configured email address, it is successfully delivered to the intended recipients. However, when someone else sends an email to the same address, it does not reach the Mailman application, and no error messages are generated. I have already tried the following troubleshooting steps: 1. Checked the Mailman container logs: I reviewed the logs using the docker logs command, but I did not find any error messages or warnings related to the incoming email issue. 2. Verified the Mailman installation: I ensured that Mailman is correctly installed and configured according to the provided documentation. All dependencies are met, and the outgoing email functionality is working as expected. 3. Checked the Mailman configuration: I reviewed the configuration files of Mailman and ensured that all necessary settings are properly configured. However, I couldn't identify any specific variables or options related to incoming email handling. Despite these efforts, I am still unable to receive incoming emails in the Mailman application. I would greatly appreciate any guidance or assistance you can provide to help resolve this issue. Please let me know if there is any additional information or logs that would be helpful for troubleshooting. Thank you in advance for your time and support.

2 1

social login added, but can't connect new login to existing users
by Jörg Schulz Sept. 6, 2023

Sept. 6, 2023

after lots of research in this list, we managed to add a keycloak login to mailman3. MAILMAN_WEB_SOCIAL_AUTH = ["allauth.socialaccount.providers.keycloak"] # OIDC_CREATE_USER = False SOCIALACCOUNT_PROVIDERS = { 'keycloak': { 'KEYCLOAK_URL': 'https://keycloak.my.domain', 'KEYCLOAK_REALM': 'nct' # 'SCOPE': ['profile', 'email', 'preferred_username'], } } and some entries in https://mailman.my.domain/admin/socialaccount/socialapp This works perfectly for non-existing users/emails. They are created and they are able to subscribe to lists. But, when a user already exists, this does not work; the user gets a password reset email like "...an account using that email address already exists. In case you have forgotten about this, please use the password forgotten procedure to recover your account:..." Which kind of setting/tweak is needed to associate social login accounts to existing mail addresses?

4 4

Postorius no connection to REST API
by rrosner5＠gmail.com Sept. 5, 2023

Sept. 5, 2023

Hello, I've seen some similar problems, but have not yet found anything that would work for me. I just installed Mailman3-full from debian stable repository (Mailman 3.2.1 to be precise). Mailman itself is working, also are all mailing lists imported from Mailman 2.1. But I just can not get the web interface of Postorius to work. I did manage to get it running through apache2 after I figured out what I had to remove from the old site config and replace it with the stuff in the automatically created config. Upon loading the site, I was able to log in and verify the mail address, but it kept showing that it wasn't able to connect to the REST API. I double and triple checked, that mailman-web.py and mailman-hyperkitty.cfg contain the same API key (api_key in hyperkitty and SECRET_KEY and MAILMAN_ARCHIVER_KEY in mailman web), but that didn't help. I also saw a recommendation, that it can cause problems, when the API key uses symbols like / so I switched it to _ in all three places, but that didn't help. The configuration for mailman, mailman-web and hyperkitty all are set to using https, switching them to http doesn't solve any of the problems. Meanwhile, the site only shows "Internal Server Error" even though I did not change anything. But probably not that relevant while it can't be used either way. What I can provide on logs: /usr/share/mailman3-web/manage.py mmclient Python 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>> client.system Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 95, in call response, content = Http().request(url, method, data_str, headers) File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1513, in request (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1263, in _request (response, content) = self._conn_request(conn, request_uri, method, body, headers) File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1186, in _conn_request conn.connect() File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1049, in connect raise socket_err File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1010, in connect sock.connect((self.host, self.port)) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/lib/python3/dist-packages/mailmanclient/client.py", line 74, in system return self._connection.call('system/versions')[1] File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 109, in call raise MailmanConnectionError('Could not connect to Mailman API') mailmanclient.restbase.connection.MailmanConnectionError: Could not connect to Mailman API From /var/log/mailman3/mailman.log of the last restart: Aug 08 15:28:56 2021 (5322) in runner caught SIGTERM. Stopping. Aug 08 15:28:56 2021 (5322) in runner exiting. Aug 08 15:28:57 2021 (5322) in runner caught SIGTERM. Stopping. Aug 08 15:28:57 2021 (5321) command runner caught SIGTERM. Stopping. Aug 08 15:28:58 2021 (5321) command runner exiting. Aug 08 15:28:57 2021 (5320) bounces runner caught SIGTERM. Stopping. Aug 08 15:28:58 2021 (5320) bounces runner exiting. Aug 08 15:28:57 2021 (5319) archive runner caught SIGTERM. Stopping. Aug 08 15:28:57 2021 (5323) lmtp runner caught SIGTERM. Stopping. Aug 08 15:28:58 2021 (5323) lmtp runner exiting. Aug 08 15:28:58 2021 (5319) archive runner exiting. Aug 08 15:28:57 2021 (5329) virgin runner caught SIGTERM. Stopping. Aug 08 15:28:57 2021 (5330) digest runner caught SIGTERM. Stopping. Aug 08 15:28:58 2021 (5330) digest runner exiting. Aug 08 15:28:58 2021 (5329) virgin runner exiting. Aug 08 15:28:58 2021 (5324) nntp runner caught SIGTERM. Stopping. Aug 08 15:28:58 2021 (5318) Master watcher caught SIGTERM. Exiting. Aug 08 15:28:59 2021 (5318) Master watcher caught SIGTERM. Exiting. Aug 08 15:28:59 2021 (5323) lmtp runner caught SIGTERM. Stopping. Aug 08 15:28:59 2021 (5324) nntp runner caught SIGTERM. Stopping. Aug 08 15:28:59 2021 (5320) bounces runner caught SIGTERM. Stopping. Aug 08 15:28:59 2021 (5329) virgin runner caught SIGTERM. Stopping. Aug 08 15:28:59 2021 (5324) nntp runner exiting. Aug 08 15:28:59 2021 (5326) pipeline runner caught SIGTERM. Stopping. Aug 08 15:28:59 2021 (5319) archive runner caught SIGTERM. Stopping. Aug 08 15:28:59 2021 (5330) digest runner caught SIGTERM. Stopping. Aug 08 15:28:59 2021 (5325) out runner caught SIGTERM. Stopping. Aug 08 15:28:59 2021 (5328) retry runner caught SIGTERM. Stopping. Aug 08 15:29:00 2021 (5327) rest runner caught SIGTERM. Stopping. Aug 08 15:28:58 2021 (5327) rest runner caught SIGTERM. Stopping. Aug 08 15:29:01 2021 (5327) rest runner caught SIGTERM. Stopping. Aug 08 15:29:01 2021 (5325) out runner exiting. Aug 08 15:29:03 2021 (5326) pipeline runner exiting. Aug 08 15:29:03 2021 (5327) rest runner exiting. Aug 08 15:29:11 2021 (5328) retry runner exiting. Aug 08 15:29:48 2021 (5318) Master stopped Aug 08 15:30:30 2021 (5545) Master started Aug 08 15:30:34 2021 (5546) archive runner started. Aug 08 15:30:34 2021 (5549) in runner started. Aug 08 15:30:34 2021 (5550) lmtp runner started. Aug 08 15:30:34 2021 (5554) rest runner started. Aug 08 15:30:35 2021 (5553) pipeline runner started. Aug 08 15:30:35 2021 (5552) out runner started. Aug 08 15:30:36 2021 (5551) nntp runner started. Aug 08 15:30:36 2021 (5547) bounces runner started. Aug 08 15:30:36 2021 (5556) virgin runner started. So everything seems to be starting smoothly. An excerpt from /var/log/mailman3/web/mailman-web.log: Traceback (most recent call last): File "manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 364, in execute_from_\ command_line utility.execute() File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 308, in execute settings.INSTALLED_APPS File "/usr/lib/python3/dist-packages/django/conf/__init__.py", line 56, in __getattr__ self._setup(name) File "/usr/lib/python3/dist-packages/django/conf/__init__.py", line 41, in _setup self._wrapped = Settings(settings_module) File "/usr/lib/python3/dist-packages/django/conf/__init__.py", line 110, in __init__ mod = importlib.import_module(self.SETTINGS_MODULE) File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/usr/share/mailman3-web/settings.py", line 486, in <module> from settings_local import * File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 724, in exec_module File "<frozen importlib._bootstrap_external>", line 859, in get_code File "<frozen importlib._bootstrap_external>", line 916, in get_data PermissionError: [Errno 13] Permission denied: '/usr/share/mailman3-web/settings_local.py' [uwsgi-daemons] throttling "python3 manage.py qcluster" for 25 seconds not sure where the Permission denied comes from/what permissions should be. /usr/share/mailman3-web/settings_local.py is a symlink owned by root to /etc/mailman3/mailman-web.py owned by list:list with 640 permission. The apache2 error log contains nothing from the last restart of mailman3 and mailman3-web and trying to load the web page, the only interesting lines to the end of the log are [Sun Aug 08 14:40:33.642119 2021] [proxy:error] [pid 2248:tid 140583379425024] (111)Connection refused: AH02454: uwsgi: attempt to connect to Unix domain socket /run/mailman3-web/uwsgi.sock (localhost) failed [Sun Aug 08 14:40:33.644627 2021] [:error] [pid 2248:tid 140583379425024] [client 134.61.97.39:11893] AH10101: failed to make connection to backend: httpd-UDS:0 [Sun Aug 08 14:54:22.555229 2021] [proxy:error] [pid 2248:tid 140583379425024] (111)Connection refused: AH02454: uwsgi: attempt to connect to Unix domain socket /run/mailman3-web/uwsgi.sock (localhost) failed [Sun Aug 08 14:54:22.555293 2021] [:error] [pid 2248:tid 140583379425024] [client 134.61.97.39:11913] AH10101: failed to make connection to backend: httpd-UDS:0 [Sun Aug 08 14:58:24.376278 2021] [mpm_event:notice] [pid 2247:tid 140583610946688] AH00492: caught SIGWINCH, shutting down gracefully [Sun Aug 08 14:58:34.534966 2021] [mpm_event:notice] [pid 3696:tid 140070763447424] AH00489: Apache/2.4.48 (Debian) OpenSSL/1.1.1d mod_wsgi/4.6.5 Python/3.7 configured -- resuming normal operations [Sun Aug 08 14:58:34.535159 2021] [core:notice] [pid 3696:tid 140070763447424] AH00094: Command line: '/usr/sbin/apache2'

7 31

Postfix-Fehler mit mailman-web
by Jens Günther Sept. 4, 2023

Sept. 4, 2023

Hello, I installed mailman with Docker already a while ago, now running v0.4.5 with: Mailman Core-Version 3.3.8 (Tom Sawyer) Mailman Core API-Version 3.1 Mailman Core Python-Version 3.10.9 Postorius Version 1.3.8 Already a while I get this type of error: 2023-08-26T08:21:59.517986+02:00 postfix/smtpd[2843414]: connect from unknown[192.168.176.6] 2023-08-26T08:21:59.522238+02:00 postfix/smtpd[2843414]: NOQUEUE: reject: RCPT from unknown[192.168.176.6]: 450 4.7.25 Client host rejected: cannot find your hostname, [192.168.176.6]; from=<$admin(a)mydomain.de> to=<$recipient> proto=ESMTP helo=<mailman-web> 2023-08-26T08:21:59.575145+02:00 postfix/smtpd[2843422]: connect from unknown[192.168.176.6] 2023-08-26T08:21:59.581227+02:00 postfix/smtpd[2843422]: NOQUEUE: reject: RCPT from unknown[192.168.176.6]: 450 4.7.25 Client host rejected: cannot find your hostname, [192.168.176.6]; from=<root(a)$mailman.mydomain.de> to=<root@localhost> proto=ESMTP helo=<mailman-web> 2023-08-26T08:21:59.581548+02:00 postfix/smtpd[2843422]: disconnect from unknown[192.168.176.6] ehlo=1 mail=1 rcpt=0/1 rset=1 quit=1 commands=4/5 Where - $admin(a)mydomain.de is set in MAILMAN_ADMIN_EMAIL (mm-web) and as site_owner in mailman-extra.cfg (mm-core) and as DEFAULT_FROM_EMAIL in settings_local.py (mm-web) - $mailman.mydomain.de is set in SERVE_FROM_DOMAIN (mm-web) For me it seems, that mm-web-container is trying to send mail, but don't know why and which ... I can't reproduce it. Does anybody have a clue on wehere I can have a look what happens ... and how to change it? Thank you, Jens.

3 8

Mailman REST API not available. Please start Mailman core.
by joseverde＠gmail.com Sept. 4, 2023

Sept. 4, 2023

Okay, so I got the webservice of Mailman3 up and running, but when I go to mailman3/postorius/lists/, I get the following error: Something went wrong Mailman REST API not available. Please start Mailman core. When I logged in to the admin user, it threw a server 500 error, but I could refresh and see myself logged in. If I try to go to lists or mailman settings, however, I get the error above.

4 15

How to debug DMARC check?
by Gilles Filippini Sept. 3, 2023

Sept. 3, 2023

Hi, I've been running a dockerized mailman3 server for several years with DMARC mitigation set to conditional. I use the 0.4 version of the mailman3 docker images. Recently I had to update my network configuration to prevent spammers using IPv6 to circumvent the no-relay configuration of my exim4 server. Previously, I had both exim4 and mailman3 on the same docker bridge 172.19.0.0/16 and exim4 was configured to relay email from this network only. There was no problem regarding DMARC mitigation : most of the emails didn't need to be altered. Now I have added an internal docker network 172.29.0.0/16 and configured exim4 to relay from this internal network only. Since this network is not accessible from the host, IPv6 spammers are blocked (*). But since this modification mailman alters every emails with DMARC mitigation. Any hint how to debug this issue? Thanks in advance, _g. (*) With IPv6 incoming connexions mailman docker replaces the client address with the IPv4 address of the host one the bridge network: 172.19.0.1.

2 4

Add date stamps to django (I think) logfiles?
by kjohnson＠eclypse.org Sept. 1, 2023

Sept. 1, 2023

The webserver and archive part of mailman writes some logfiles on my system with entries like: 17:19:05 [Q] INFO Process-1 guarding cluster twenty-pluto-arizona-pasta 17:19:05 [Q] INFO Process-1:4 pushing tasks at 2831 17:19:05 [Q] INFO Q Cluster twenty-pluto-arizona-pasta running. These seem to match this format in settings.py: 'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s' except that I would expect asctime to have a date attached. A pointer to relevant documentation would be very much appreciated. Thanks. Ken

3 2

Installing Mailman3 on Debian Bullseye
by Franklin Weng Aug. 30, 2023

Aug. 30, 2023

Hi, After several times attempts I finally successfully build a (reliable enough) mailman3 instance on Debian. I had a blog article describing how I did it. Maybe there were something you find it ridiculous, but they are all what I found important and may not be shown in the current documents. https://blogs.slat.org/node/1012 Hope this helps. Franklin

6 9

log rotation in maxking-docker-version
by Jens Günther Aug. 27, 2023

Aug. 27, 2023

Hey there, while checking some logs I was realising that there is no log rotation and some of them piled up to 5GB over the past year(s). Does anybody know how to activate it (if it's possible) or ... Ok, I found a clue and it's still open :-) https://github.com/maxking/docker-mailman/issues/535 -> does anybody have a different approach? Regards, Jens.

2 3

Can't start mailman after reboot.
by mrl＠psfc.mit.edu Aug. 24, 2023

Aug. 24, 2023

I have successful rebooted my computer server several times, and mailman has always started without a problem Now mailman won't start. See below. I tried to see if it was a problem with mysql. But I can successfully connect to the sql server using the mailman username and password, from the command line. What else can I try??? Thanks. - Mark Aug 24 12:52:17 psfcmail2 systemd[1]: Starting GNU Mailing List Manager... Aug 24 12:52:18 psfcmail2 mailman[28151]: Traceback (most recent call last): Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/bin/mailman", line 8, in <module> Aug 24 12:52:18 psfcmail2 mailman[28151]: sys.exit(main()) Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__ Aug 24 12:52:18 psfcmail2 mailman[28151]: return self.main(*args, **kwargs) Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1055, in main Aug 24 12:52:18 psfcmail2 mailman[28151]: rv = self.invoke(ctx) Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/lib/python3.10/site-packages/mailman/bin/mailman.py", line 69, in invoke Aug 24 12:52:18 psfcmail2 mailman[28151]: return super().invoke(ctx) Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke Aug 24 12:52:18 psfcmail2 mailman[28151]: return _process_result(sub_ctx.command.invoke(sub_ctx)) Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke Aug 24 12:52:18 psfcmail2 mailman[28151]: return ctx.invoke(self.callback, **ctx.params) Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke Aug 24 12:52:18 psfcmail2 mailman[28151]: return __callback(*args, **kwargs) Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func Aug 24 12:52:18 psfcmail2 mailman[28151]: return f(get_current_context(), *args, **kwargs) Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/lib/python3.10/site-packages/mailman/commands/cli_control.py", line 91, in start Aug 24 12:52:18 psfcmail2 mailman[28151]: status, lock = master_state() Aug 24 12:52:18 psfcmail2 mailman[28151]: File "/opt/mailman/venv/lib/python3.10/site-packages/mailman/bin/master.py", line 104, in master_state Aug 24 12:52:18 psfcmail2 mailman[28151]: os.kill(pid, 0) Aug 24 12:52:18 psfcmail2 mailman[28151]: PermissionError: [Errno 1] Operation not permitted Aug 24 12:52:19 psfcmail2 mariadbd[22160]: 2023-08-24 12:52:19 3191 [Warning] Aborted connection 3191 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets)

2 3

Only allow users to subscribe from certain domains
by bob B Aug. 23, 2023

Aug. 23, 2023

Does Mailman have anyway to just allow users to subscribe themselves to a list from certain domains... ie any user from test.com or test2.com I do not believe this exists, but have a list manger who is asking. I feel they will have to either, approve all requests, or have a web frontend built that could filter that out and then send the subscribe commands in the backend. Thanks

2 2

applying patch 1130
by David Newman Aug. 22, 2023

Aug. 22, 2023

Please forgive a dumb question from a non-programmer: How to apply the !1130.diff patch to a venv installation? https://gitlab.com/mailman/mailman/-/merge_requests/1130 I am looking to upgrade Debian 11 to 12. The latter uses python3.11, which no longer has a dependency on importlib_resources. As a result, MM3 won't start unless all references to importlib_resources instead call importlib.resources, now part of stdlib. I tried patching all importlib.resource references by hand, and MM3 starts – but that can't be the "right" way to do this. Also, the diff references a src directory that doesn't exist under /opt/mailman. Thanks in advance for patching clues. dn

3 5

importlib_resources issue?
by Dave McGuire Aug. 21, 2023

Aug. 21, 2023

Hi folks, I'm trying to test a fresh venv-based install from this procedure: https://docs.mailman3.org/en/latest/install/virtualenv.html Platform is SmartOS (Solaris 11) with Python 3.10. The installation goes fine, but initial startup bombs with: ImportError: cannot import name 'path' from 'importlib_resources' (/opt/mailman/venv/lib/python3.10/site-packages/importlib_resources/__init__.py) I assume this is related to importlib_resources moving into stdlib at some point. Is there a reasonable way around this? Thanks, -Dave -- Dave McGuire, AK4HZ New Kensington, PA

2 2

Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
by Mail Person Aug. 21, 2023

Aug. 21, 2023

Hello everyone. I am trying to sort out an error happening when messages are sent to the mailman server. The emails are being sent to the list but I get this error each time: Internal Server Error: /mailman3/hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1") Request Method: POST Request URL: http://localhost/mailman3/hyperkitty/api/mailman/archive?key=7OF5IogiQItSUK… Django Version: 2.2.12 Python Executable: /usr/bin/uwsgi-core Python Version: 3.8.10 Python Path: ['.', '', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Thu, 17 Aug 2023 17:13:09 +0000 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 71. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 209. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 315. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 226. _mysql.connection.query(self, query) The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1")) was the direct cause of the following exception: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py" in inner 40. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in _decorator 71. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py" in wrapped_view 54. return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in archive 117. add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in add_to_list 153. email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save 740. self.save_base(using=using, force_insert=force_insert, File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save_base 777. updated = self._save_table( File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _save_table 870. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _do_insert 907. return manager._insert([self], fields=fields, return_id=update_pk, File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method 82. return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py" in _insert 1186. return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py" in execute_sql 1375. cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 67. return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute_with_wrappers 76. return executor(sql, params, many, context) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__ 89. raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 71. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 209. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 315. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 226. _mysql.connection.query(self, query) Exception Type: OperationalError at /hyperkitty/api/mailman/archive Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1") Request information: USER: AnonymousUser

2 2

Ansible role to install Mailman3
by samuel.d.darwin＠gmail.com Aug. 4, 2023

Aug. 4, 2023

Hi, Before installing the Mailman Suite, I searched on https://galaxy.ansible.com/ for Ansible roles that might automate the whole process. https://github.com/galaxyproject/ansible-mailman3 turned out to be quite good. However, it was slightly out-of-date. I cloned it, made a few bug fixes, and corrected linting errors. Then, going further, added optional basic installations of Nginx, Postfix, PostgreSQL into the role. If anyone is installing Mailman they are welcome to try it out. https://github.com/cppalliance/ansible-mailman3 Now, I see there was a discussion on this very list about https://github.com/rivimey/ansible-mailman3 which was also a fork of natefoo's repo. What are the differences? As Stephen Turnbull wrote: "her (rivimey's) setup may be more complicated than most folks need (IIRC, she has a multi-homed setup with Mailman core, Postorius, and HyperKitty on three separate IPs on a separate subnet in the DMZ, while Mailman was really designed for three Mailman applications running on the same host, along with the database server and MTA)." My version of the role didn't follow the tactic of "three separate IPs on a separate subnet in the DMZ". Mainly, it adds postfix and postgres. Fixes small errors. Also, in a git branch of the repository named mailman3-core, I removed all the web code, so there is a version of the role that only installs core. We will use that while bundling the web components into another Django app, already deployed with docker. There is an open question about maintenance. Among the three roles mentioned here, which is being actively kept up-to-date? Perhaps there are no guarantees from any of them. At this instant in time, https://github.com/cppalliance/ansible-mailman3 is the most recent. Would it make sense to mention these roles in the official documentation? Not sure. Installing mailman3 is complicated and because of that, is a good fit for Ansible.

3 2

Mailman3 core didn't start after reboot, because mysql wasn't running yet.
by Mark London July 29, 2023

July 29, 2023

Hi - We had a power outage, and after my Ubuntu server rebooted , mailman 3 tried to start before mysql was running. I guess because it wasn't a clean reboot, the mysql server took much longer than normal to start, while checking databases. So mailman3 couldn't start. Normal reboots never had this problem. Should it have kept trying? In any event, mailman3 core didn't start, but mailman3-web did start. But it wasn't very happy, of course. And any time someone tried to access the mailing list website, it would send me an error message. I had hundreds of such emails in the morning., that started with the one below. Is there a way to configure my system to make sure this doesn't happen again? Thanks. - Mark Service Unavailable: /mailman3/ Report at /mailman3/ Service Unavailable: /mailman3/ Request Method: GET Request URL:https://lists.psfc.mit.edu/mailman3/ Django Version: 4.1.8 Python Executable: /opt/mailman/venv/bin/uwsgi Python Version: 3.10.6 Python Path: ['.', '', '/riusMiddleware')

4 6

Something wrong in Django admin interfaces
by Franklin Weng July 21, 2023

July 21, 2023

Hi, Since both Mark and Danil kindly told me that my wish to disable web anonymous account registering requires new version of mailman and django-mailman, I found that newest Debian bookwarm (12) has * mailman 3.3.8-1 (including mailman3-full and mailman3-web 20200530-2.1) * python3-django-mailman3 : 1.3.9-1 * python3-mailman-hyperkitty : 1.2.1 So I re-installed mailman3 (and took notes so that I could write a blog). After installing that, I found that it has problems in django admin page. When I clicked any features like users, Mail domains, Failed tasks... anyone, no matter New or Change, the web page was covered by the first page. All the configuration fields/labels are covered by the first page. Please see the screenshot: https://nextcloud.slat.org/index.php/s/e5KKEEbqDK6DpiN https://nextcloud.slat.org/index.php/s/nLgMRQ9ACyxCwoc It was not able to do the administration tasks. I looked at django-mailman issues https://gitlab.com/mailman/django-mailman3/-/issues but didn't find anything useful. Does anyone have similar issues? What can I do to solve this? Thanks, Franklin

2 1

Any possible way to reduce robot spam accounts?
by Franklin Weng July 21, 2023

July 21, 2023

Hi, I just installed a mailman3 instance, even haven't announced to anyone yet, there are a lot of spam robot accounts registering into my system. (See https://nextcloud.slat.org/index.php/s/WWHo7F3igYxn8LY for screenshot) I really have no idea how the robots find my system since I haven't told anyone the ip or URL of my mailman instance. And, they're registering on the system, not the mailing list since I have only one list for testing now and it wasn't assaulted by spam yet. No matter what, what I want to ask is that, is there a way to either * just let users join from email (lists-join(a)lists.url) instead of registering on the mailman instance? or * use a scheme like CAPTCHA to reduce the successful opportunity of robot registering? Any help is appreciated! Franklin

4 7

List mails blocked from GMX - web.de > Spamfolder
by itservice＠knut-krueger.de July 19, 2023

July 19, 2023

How can I configure Mailman3 so that Mails from the list will not be blocked by gmx.net I tried to send with the same user (mailman.cf) at my mail server, which will not be blocked by gmx. unfortunately, I get no bounce back message. They do just not appear in the mail folder of gmx. At web.de they go to the SPAM folder.

4 4

Import old mails into list
by franklin＠slat.org July 19, 2023

July 19, 2023

Hi, I just built a new mailman3 instance. I installed directly on Debian using mailman3-full and follow /usr/share/doc/mailman3/README.Debian to set up database/web. Then I'd like to import some old mail (in a mbox file) into this list. I searched for any documentation but all I got is Import list data (from 2.1): https://docs.mailman3.org/projects/mailman/en/release-3.0/src/mailman/comma… How do I import an archive into a new mailing list: https://wiki.list.org/DOC/How%20do%20I%20import%20an%20archive%20into%20a%2… The first is import list data, not list contents (mails). The second is very old, written in 2007, and now I even can't find bin/arch commands. Also, this post in this mailing list: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… Which nobody replied yet. And when I tried `django-admin hyperkitty_import` it even told me "Unknown command: 'hyperkitty_import'". Could anyone please tell me how to import old mails from mbox file into a new mailing list? Thanks, Franklin

2 1

Configure administrative users via command line?
by kjohnson＠eclypse.org July 18, 2023

July 18, 2023

I recently migrated a mailing list from Mailman 2.x to Mailman 3.x. Sending messages to and receiving messages from the list seems to work well. The web interface archives seem to be working well. Xapian is configured as the search engine. With only 60K some messages, archive search seems very fast. How can I configure administrative users via command line? I need to process a moderation request. I have root access on the server. A pointer to the appropriate documentation would be more than welcome. Thank you to anyone who can provide help. Ken

2 3

where do I find the text of a bounce email ?
by Per Jessen July 1, 2023

July 1, 2023

All, I am trying to figure why a particular outbound message was bounced, so I'm looking for the text of the NDR that was sent - where do I find that ? Fyi, the outbound message was accepted by the first relay, and then presumably later bounced by the actual recipient server (Office365). Any and all hints much appreciated. -- Per Jessen, Zürich (4.6°C) Member, openSUSE Heroes

3 5

Viewing/editing bounce score/status in Postorius
by Tom Smyth July 1, 2023

July 1, 2023

Is this possible? As of 1.3.2 I can't find it. Perhaps it's in a newer version? I'm working on upgrading. Thanks! -- Tom Smyth (he/him)

2 1

Hyperkitty list overview error: thread matching query does not exist
by Darren Hart June 29, 2023

June 29, 2023

When attempting to visit the hyperkitty archives list overview for one of my two lists: Https://lists.mydomain.com/hyperkitty/list/mylist@mydomain.com/ The area where the threads should appear shows the loading spinner (but the activity summary bar chart and most active posters post counts do eventually load). The following appears in /var/log/mailman3/web/mailman-web.log: ERROR 2022-05-23 17:50:50,201 1694 django.request Internal Server Error: /hyperkitty/list/mylist(a)mydomain.com/posted-to Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/lib/view_helpers.py", line 135, in inner return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mlist.py", line 251, in overview_posted_to for thread in mlist.recent_threads: File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py", line 142, in recent_threads return self.cached_values["recent_threads"]() File "/usr/lib/python3/dist-packages/hyperkitty/models/common.py", line 59, in __call__ return self.get_or_set(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py", line 281, in get_or_set return [Thread.objects.get(pk=pk) for pk in thread_ids] File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py", line 281, in <listcomp> return [Thread.objects.get(pk=pk) for pk in thread_ids] File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 406, in get raise self.model.DoesNotExist( hyperkitty.models.thread.Thread.DoesNotExist: Thread matching query does not exist. Viewing “All Threads” or Threads by month (any month) work as expected, displaying the appropriate threads. The archives for this list worked previously. There was an incident that resulted in DNS not working and the archiver failing for a few days. The mail that came in during that time does show up in the other views. The hyper kitty archives for a second list on this Mailman3 instance is working correctly. How can I fix this view of the archives? Or how can I get more information on what might be failing? Thanks, Darren

3 6

archives html user interface menu selection retention after refresh
by Ken Alker June 29, 2023

June 29, 2023

I have just started studying the archives feature and I've noticed that if I select "Most Active" in the top menu and then I refresh my browser, the refreshed screen jumps back to Recent after the refresh. In fact, if ANY of the five menu items across the top are selected and then a browser refresh is performed, the result is that one is always taken back to the Recent menu item. It was not apparent to me for a couple of days that this was happening and I'd convinced myself there was some bug in the Recent vs. Most Active lists because the Most Active list changed entirely after I did a refresh, when, in fact, I was then viewing the Recent list again. Confusing. Is there some way that whatever menu item was being viewed before a refresh could be retained so that the same menu item is shown after a refresh? Thinking this through even further, I suppose a side-affect of this request could be that visiting the same page much later (ie. not immediately after a refresh) may result in the page NOT defaulting to the Recent menu item any longer (unless once could clear the "last menu item" memory when the viewer moves AWAY from or closes the page). I don't have enough familiarly with the interface to suggest if this would be a bad side-effect, or good, but would this, indeed become a side-affect that would have to be thought through?

2 1

Re: How do I specify the TCP port number for Postgresql?
by kjohnson＠eclypse.org June 28, 2023

June 28, 2023

Thank you for this helpful and prompt reply! Ken

1 0

How do I specify the TCP port number for Postgresql?
by kjohnson＠eclypse.org June 28, 2023

June 28, 2023

The mailman.cfg file on the system I am working on includes this line: url: postgres://mailman3:mailman3:@localhost/mailman3 This works fine as long as the postgresql server is at the default port number of 5432. How do I specify a different TCP port number? My guess is that a change to this line is needed, but unfortunately, it is not obvious to me how to make that change, and I have been unable to find this information in the mailman3 documentation or in a previous mailing list post. Feel free to point me at the documentation or post that I have overlooked. Thanks, Ken

2 1

Archive date/time vs elapsed time
by Mark June 28, 2023

June 28, 2023

Hi In the archive's "Recent", "Most active" etc displays, the date and time of each post is shown. However, when viewing by "Thread" listing, each post shows the elapsed time since it was posted ie. the number of days, hours since it was posted (eg. 1 day, 2 hours). Is it possible to set Hyperkitty to display the "Thread" listing with the date/time in place of that elapsed time? Thanks Mark

2 4

Postorius claims 2 messages held when there are none - out of sync
by Ken Alker June 27, 2023

June 27, 2023

When I view a list via Postorius the "Held Messages" menu item has an inverse-red-"2" next to it, as if there are two held messages. There are, in fact, no messages in "/opt/mailman/mm/var/queue/*" (assuming the held messages should be in "pipeline" subdirectory). When I click on "Held Messages" I get "Something went wrong" followed by "HTTP Error 500: {"title": "500 Internal Server Error"}". How do I fix this?

3 8

Message is received but is not sent out. Another, similar message, is processed fine.
by Allan Hansen June 27, 2023

June 27, 2023

Hi all, One of my subscribers is trying to send a message to a list, but it is not being posted. The smtp.log shows that her first message was posted, but a second message (with 2 retries) was not: mailman@list:~/logs$ grep herName smtp.log Jun 18 11:22:05 2023 (1243316) ('127.0.0.1', 56686) >> b'MAIL FROM:<herName(a)hotmail.com> SIZE=40109' Jun 18 11:22:05 2023 (1243316) ('127.0.0.1', 56686) sender: herName(a)hotmail.com Jun 18 11:25:01 2023 (1243318) <CH3PR12MB82593510B90D8F05177C606BBE5EA(a)CH3PR12MB8259.namprd12.prod.outlook.com> post to theList(a)domain.tlc from herName(a)hotmail.com, 40061 bytes Jun 18 11:52:42 2023 (1243316) ('127.0.0.1', 48448) >> b'MAIL FROM:<herName(a)hotmail.com> SIZE=18343' Jun 18 11:52:42 2023 (1243316) ('127.0.0.1', 48448) sender: herName(a)hotmail.com Jun 18 19:01:04 2023 (1243316) ('127.0.0.1', 59550) >> b'MAIL FROM:<herName(a)hotmail.com> SIZE=19123' Jun 18 19:01:04 2023 (1243316) ('127.0.0.1', 59550) sender: herName(a)hotmail.com Jun 26 17:18:11 2023 (1243316) ('127.0.0.1', 44732) >> b'MAIL FROM:<herName(a)hotmail.com> SIZE=18930' Jun 26 17:18:11 2023 (1243316) ('127.0.0.1', 44732) sender: herName(a)hotmail.com At the second retry she copied me and I saw nothing untowards in her message. She's not getting any error return messages and her subscription is OK. Both messages were multipart/alternative and both contained Hebrew text. I told her I'd look at this, but I'm at a loss. Yours, Allan

3 6

Problems after upgrading from bullseye to bookworm
by Eggert Ehmke June 27, 2023

June 27, 2023

I upgrade my server to bookworm. In the process python was upgraded from 3.7 to 3.9 . I reinstalled the virtualenv and the mailman packages. When I now start mailman-web, I get this in web/logs/uwsgi-error.log: pkg_resources.DistributionNotFound: The 'flufl.lock>=4.0' distribution was not found and is required by hyperkitty What did I miss? ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Replaced multipart/alternative part with first alternative.

5 5

attachment.html
by Sam Darwin June 27, 2023

June 27, 2023

Hi, Using a typical modern email client such as Gmail, the messages are usually html formatted. Then, Hyperkitty displays attachment.html on every message. Can that be removed? I was searching for the topic "attachment.html" in the archives and there wasn't much information. What finally cracked the code was Mark's comment: If in Postorius on the list's Settings -> Alter Messages page, Filter Content and Convert html to plaintext are both Yes, the HTML should be converted to plain text and that should appear in HyperKitty. It turns out "Filter content" affects many settings. It's easy to skip over that. I wonder if the UI might call attention somehow, with a graphic element, or with text: "Settings on this page will not take effect. Because 'Filter content' has not been enabled". It is right that "Filter content" must be set in order to remove attachment.html ? And now, a search will have more results. :-)

2 3

clicking on a username when viewing a thread results in "Page not found"
by Ken Alker June 27, 2023

June 27, 2023

If I click on a username when viewing a thread, it returns "Page not found" then "This page either doesn't exist, or it moved somewhere else.". Any idea why this may be happening? Also, why is it that some usernames are not clickable (ie. black text with no hyperlink)? Thanks! Ken

3 4

old usernames appearing in mailman.log
by Ken Alker June 27, 2023

June 27, 2023

I have just migrated from a mailman 3.2.2 Debian package installation to a mailman 3.3.8 source install (in a Python virtual environment as mailman user) on an Ubuntu 20.04.6 LTS system. Several months ago, the system had been upgraded from mailman V2 to mailman V3.2.2. The server has been active for over 20 years. I've just taken on the project so I'm new to all of this, and one thing I noticed that caught my eye, and has left me curious, is that for the first a couple of seconds starting just after midnight every night (just after log rotation) mailman.log would have many (dozens, or hundreds) of lines entered in it just like this one: Jun 19 00:00:04 2023 (2266) 127.0.0.1 - - "GET /3.0/users/old-user(a)domain.tld HTTP/1.1" 404 58 where "old-user(a)domain.tld" is the email address of a user who either had unsubscribed or changed their email address (I don't know which, I just know they are no longer a member/subscriber to the list). Each successive entry in the log would have a different "old username" in it. I don't understand why these entries would be in the log and what they mean. Can someone shed some light on this for me? This is merely a question of curiosity, and not one of import. (Since migrating from 3.2.2 to 3.3.8 this behavior has stopped, be that GOOD or BAD.) Ken

2 8

Global ban using regex
by Henrik Rasmussen June 27, 2023

June 27, 2023

Using some regex testers and generators like https://regex101.com and https://www.site24x7.com/tools/regex-parser.html, the regex (.*)(a)inker\.azerbaijan\.su seems to match EVERYTHING(a)inker.azerbaijan.su (EVERYTHING is constantly changing in current spam mails). However, entering the regex to the Global Ban Add ban /postorius/bans/ result in: `An error occurred: Invalid Parameter "email": Expected a valid email address or regular expression, got (.*)(a)inker\.azerbaijan\.su.'` Same result when using [Ban list] in postorius/lists/[LISTNAME]/bans/. Entering a none regex addresses like eknejfj(a)inker.azerbaijan.su works fine. Does Mailman use a different regex syntax I am using GNU Mailman 3.3.4 (Tom Sawyer) and Mailman Core Python Version 3.8.10 (default, May 6 2021, 06:30:44) [GCC 9.3.0] (maxking/mailman-core:0.3 and maxking/mailman-web:0.3)

2 1

starting gunicorn per mailman.log yet gunicorn not installed
by Ken Alker June 27, 2023

June 27, 2023

I see the below in my mailman.log, yet I installed uwsgi and NOT gunicorn. My mainmanweb.service file includes "ExecStart=/opt/mailman/venv/bin/uwsgi --ini /etc/mailman3/uwsgi.ini" and not the gunicorn equivalent. Does this log entry make sense in my case? [2023-06-22 12:44:36 -0700] [411132] [INFO] Starting gunicorn 20.1.0

4 7

messages stuck in the bad queue
by Ken Alker June 26, 2023

June 26, 2023

I am working on a mail server that was migrated from mailman V2 to mailman V3 in February. I just noticed that there are over 1000 messages in the bad queue. All of the messages in the bad queue are dated 2/2/23 and all have a timestamp within a one-hour window of each other. I used mailman qfile to inspect five random emails in the bad queue and so far they are all "successfully subscribed" emails, but I presume there are no guarantees there aren't others mixed in there that might be legitimate emails (ie. I just unshunted 140 emails in the shunt queue and they were all good emails and all got processed). I figure that the easiest way to inspect these 1000 emails is just to have them re-delivered. I tried moving one from the bad queue to the shunt queue and I ran "mailman unshunt" but nothing happened. Is there a way to reprocess the bad queue? Also, what exactly is the bad queue? Thank you, Ken KA6KEN (happy field day weekend)

2 4

re-injecting emails that were not archived
by Ken Alker June 25, 2023

June 25, 2023

I have just migrated from a mailman 3.2.2 Debian package installation to a mailman 3.3.8 source install (in a Python virtual environment as mailman user) on an Ubuntu 20.04.6 LTS system. Several months ago, the system had been upgraded from mailman V2 to mailman V3.2.2. I noticed when I came onto the project last week that there were 140 messages stuck in the "shunt" queue for the two lists hosted by this server (I have no idea why and the old logs were not available and I don't recall the results of "mailman qfile", assuming that had shed some light on the reason why they were shunted). In any event, I unshunted these messages and they flowed out to the subscriber's email addresses successfully. What I was not expecting is that the system did not inject any of these emails into the archive. AFAICT, the messages are no longer anywhere on the system (but I could be mistaken). Here are my questions: 1) Why would these emails not have been injected into the archive? 2) Is there some way for me to re-inject these messages into the two lists? The only place I (THINK I) have them is in my own personal IMAP box since I'm a subscriber to the lists. 3) Can this be done without the emails being re-delivered to subscribers? Thank you, Ken

2 1

Re: migratng from 3.2.2 Debian packages to 3.3.8 source install - web server question
by Ken Alker June 25, 2023

June 25, 2023

As an aside, for anyone else who is trying to do the same (and to give some background into my procedures so far), I have been following these references: <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…> <https://docs.mailman3.org/en/latest/install/virtualenv.html#setup-virtualenv> My goal is to create the least change to file locations and configuration files as possible (I may move things into more "source-installed centric" locations once I have things working again), thus, many files are still in the original Debian package-allocated directories. That said, I was forced to move mailman3web.db to /opt/mailman/web as sqlite (and, hence, mailman-web) simply could not open the database as user mailman from my virtual environment (even with mailman3web.db file permissions set to 777 and owner.group set to mailman.mailman). So far I've gotten mailman running properly and handling the mailing lists, however, I have not gotten the web interface working again.

2 6

Importing MBOX from custom listserv to MM3?
by Thomas Ward June 25, 2023

June 25, 2023

We are looking at moving from a home grown listserv system at a client to potentially MailMan3 for the lists platform instead. We have historic thread data from the homegrown systems in mbox form. Is there a way to import that into MM3 for indexing in the archiver there so we dont need to keep the (literally massive) MBOX file? Sent from my Galaxy

2 2

Importing mbox file into MM3
by Mark June 23, 2023

June 23, 2023

Hi All I'm trying to import a large mbox file from a MM2 list into a new list on MM3. The new MM3 list has 3 test emails in its archive. # cd /usr/share/mailman3-web/ # python3 manage.py hyperkitty_import -l LISTNAME@DOMAIN /home/mark/tmp/LISTNAME.mbox Importing from mbox file /home/mark/tmp/LISTNAME.mbox to LISTNAME@DOMAIN Computing thread structure Synchronizing properties with Mailman Warming up cache The full-text search index is not updated for this list. It will not be updated by the 'minutely' incremental update job. To update the index for this list, run the Django admin command with arguments 'update_index_one_list LISTNAME@DOMAIN'. That all seems pretty good. Then I attempt to rebuild the archive. # python3 manage.py update_index_one_list LISTNAME@DOMAIN Indexing 3 emails The import or rebuild has failed as it's just rebuilding the existing archive of the 3 test messages. What am I doing wrong? Thanks Mark

2 2

Where in database are subject prefix sequences stored?
by Thomas Ward June 22, 2023

June 22, 2023

I am using numeric prefixes for a list, such that it's `[ListName-000012]`. Where would that sequence indicator be stored in the database? Postorious' database? Hyperkitty's database? I ask because I'm rebuilding a home-grown listserv solution using Mailman3 and the main requirement is the thread IDs being *properly sequential* with the current implementation, so I have to set the "next ID in sequence" to a start value. I don't see this clearly in the DB where the sequence value would be stored, so if anyone knows that'd be great to know so I can get this working. Thomas

2 2

Help needed to create filters that were working in Mailman 2.
by mrl＠psfc.mit.edu June 22, 2023

June 22, 2023

How do I make a filter that if it matches, it accepts the message and doesn't check the remaining filters? I could do this using the anti-spam filters in mailman 2, as there was a Accept option. But there is no Accept option in Mailman 3 (that I can find). I tried using "Default antispam action", but that didn't stop the rest of the filters from being applied. Thanks for any suggestions. - Mark

3 4

toward improving correctness when running on single core system ...
by Nelson Strother June 21, 2023

June 21, 2023

I recognize that most real computer systems now are equipped with multiple CPUs, usually with each CPU having multiple cores or hardware threads. But when one obtains a computer system from a service provider with a small expected load, it is likely that a virtual computer system will provide only a single core. Yes, one can obtain a multi-core virtual system, maybe for only twice the cost; but it would seem wasteful to make that choice solely to support starting and stopping the Mailman3 program when the system load at all other times is satisfied by a single core system. From my observations, mailman 3.3.8 is unreliable when running on a single core system. When `mailman start` is issued, there is a moderately high probability that one or more of the thirteen runner processes will not survive this startup phase. Depending upon the current usage of the mailman installation, one may or may not notice this fact. In either case, `mailman status` provides the same information, with no hint of missing or damaged capabilities when one or more of the runner processes are stillborn. [1] One potential improvement would be to increase the lifetime of the locks used by each of the runner processes during `mailman start`. [2] Based on a small sample of runs of `mailman start`, this change may provide correctness (i.e. I have yet to see a runner process go MIA). But it still seems unsatisfactory for availability / performance, as the elapsed wall clock time for the system to settle down and be functional and responsive is approximately 13 minutes. While this is an improvement from the 20 minute or greater delays observed before making this change, it still differs greatly from the nearly instantaneous startup time experienced on multicore / multiprocessor systems. The best "performance" I have yet obtained on this single core system is WITHOUT the change in [2] below, but by crossing abstraction boundaries and reducing the amount of time this single CPU spends futilely busy waiting for itself when it has not provided an opportunity for another process (maybe even the process currently holding the desired lock) to make any progress. [3] With this change, "only" 9 minutes elapsed wall clock time is wasted (with maybe only the middle 4 minutes at 100% CPU utilization) before the system settles down and is functional and responsive. However, ample room for improvement remains. [4] Please suggest better ideas or corrections. Nelson [1] For a few examples of these failures, see: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… [2] Here is a diff from the original mailman/database/factory.py (or .../mailman/.local/lib/python3.9/site-packages/mailman/database/factory.py if you wish to apply this to an installed system): > 25a26 > > from datetime import timedelta > 53c54,55 > < with Lock(os.path.join(config.LOCK_DIR, 'dbcreate.lck')): > --- > > with Lock(os.path.join(config.LOCK_DIR, 'dbcreate.lck'), > > timedelta(seconds=30)): which increases the delay from the default of 15 seconds to 30 seconds before another process will be empowered to break the lock being held by one of mailman's runner processes, which leads to the death of a process. [3] Here is a diff from the original .../mailman/.local/lib/python3.9/site-packages/flufl/lock/_lockfile.py : > 7a8 > > import subprocess > 338a340,344 > > if sys.platform.startswith('linux') and \ > > (len(subprocess.check_output(["lscpu", "-p"]).splitlines()) == 5): > > # there is only one processor [four lines of comments, and only one line for one CPU] > > # let another process run before joining the traffic jam for this lock > > os.system('sleep 20s') For possible similar benefits on other platforms, one should also include variations on this theme using something like e.g.: > > if sys.platform.startswith('win32') and \ > > subprocess.check_output('wmic path win32_Processor get NumberOfLogicalProcessors').strip().endswith(b'1'): I seriously doubt all (or even any?) other users of the flufl.lock package would benefit from this exact change ... but many may benefit from sleeping for e.g. a half-second when on a single core? In the case of Mailman3, sleeping for a fixed duration shorter than 20 seconds retains the original problem of broken locks and runner process death, while sleeping for a fixed duration longer than 20 seconds leads to an increase in both the wall clock delay time as well as the CPU time consumed during `mailman start`. I have not experimented with random or staggered delays, which may further reduce the number of simultaneously blocked waiters for this lock. [4] I have seen `mailman start`complete in less than 1 second wall clock time on a multicore / multiprocessor system. All of these observations made with mailman 3.3.8 as installed via pip.

5 8

How to tell who moderated a post?
by bryan.kartzman＠yu.edu June 21, 2023

June 21, 2023

Is there any way to tell which moderator of a list approved a particular post to that list?

2 3

Re: mailman3 auto-disabling of subscriptions
by Mark Sapiro June 19, 2023

June 19, 2023

On 6/17/23 17:02, Mike Wertheim wrote: > I set `remove_dkim_headers: yes` in the [mta] section of mailman.cfg. > > I'm seeing a bunch of these errors in the logs: > 2023-06-17 18:57:11 1qAbMA-005Y7W-HP ** mike.wertheim(a)gmail.com > <mailto:mike.wertheim@gmail.com> R=dnslookup_single_domain > T=remote_smtp_single_domain H=alt2.gmail-smtp-in.l.google.com > <http://alt2.gmail-smtp-in.l.google.com> [172.253.62.26]: SMTP error > from remote mail server after end of data: 550-5.7.26 Unauthenticated > email from gmail.com <http://gmail.com> is not accepted due to > domain's\n550-5.7.26 DMARC policy. Please contact the administrator of > gmail.com <http://gmail.com> domain\n550-5.7.26 if this was a legitimate > mail. Please visit\n550-5.7.26 > https://support.google.com/mail/answer/2451690 > <https://support.google.com/mail/answer/2451690> to learn about the\n550 > 5.7.26 DMARC initiative. > w13-20020a05622a190d00b003fde5babbf5si1170418qtc.14 - gsmtp Your issue is mail From: the gmail.com domain to the list and sent to gmail users is being bounced because it fails DMARC. I.e. Gmail applies a DMARC policy of p=reject to mail From: gmail.com even though it's published DMARC policy is p=none. You need to apply DMARC mitigations to this mail. Unfortunately, this requires setting DMARC Mitigate unconditionally to Yes. Mailman 2.1 has a feature to apply DMARC mitigation to messages From: specific domains regardless of any domain specific DMARC Policy. Unfortunately this hasn't been implemented in Mailman 3 yet. I just filed https://gitlab.com/mailman/mailman/-/issues/1084 for that. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 1

Where does sender postorius@ come from?
by Jens Günther June 19, 2023

June 19, 2023

I just realized, that mails for registering an account and/or resetting ones password are obviously sent from postorius@SERVE_FROM_DOMAIN. Why are they not sent from MAILMAN_ADMIN_EMAIL? The problem is: - an account is trying to register with a mailaddress from a HostEurope-server - HostEurope-Servers are doing Sender-Callouts (bad practise :-D) to verify that a mailaccount exists - mail is then bouncing, b/c those are not answered (shoud I?): status=bounced (host [80.237.138.5] said: 550-Sender verification is required but failed. - bounce then can't be processed, since mailaccount does not exist :-D status=bounced (mail for SERVE_FROM_DOMAIN loops back to myself) Am I missing something are shouldn't it be better to send those mails from MAILMAN_ADMIN_EMAIL? (I don't want to mess with these sender callouts or change anything, HostEurope has been a little bit of a pain in the ... with mailinglists and not much help at all). Thanks in advance, Jens.

3 8