February 2022 - Mailman-users - lists.mailman3.org

I did something stupid. Can i back out of it?
by William Oliver Feb. 20, 2022

Feb. 20, 2022

So... I run a small mailinglist using mailman3 that has been workiing great for a couple of months. Last week, all my users from gmail got kicked off because gmail decided that the list messages were "unsolicited email". I didn't find out about it until the users hit their mailman3 bounce limit. I *thought* would be seeing all the bounce messages, but apparently not. I didn't see any config settings for "see all the bounce messages" so i thought to myself, "Self, well maybe that's because my mailinglist username doesn't have a mailbox in /home/listname". So, I decided to try that. I created /home/listname/Maildir/* and logged in as fplist to see if it would put mail there. In order to log in as listname, I gave the user listname a password. Then the world ended. The mailinglist works OK for *new* posts, but it is now impossible to *reply* to posts. So, if I post as a member from user(a)somewhere.org to listname(a)example.com, it works. If I hit reply, with the From: line rewritten as listname(a)example.com, it fails. The error I get is: The reported error was “Error performing TLS handshake: An unexpected TLS packet was received.”. My suspicion is that there are two passwords floating around for "listname" -- one in mailman3 and the one in /etc/shadow, but I don't know. In retrospect, this was not my finest moment. But is there an easy way to back out of this? Thanks! billo

2 8

certain msgs not archived
by Sandor Dibuz Feb. 18, 2022

Feb. 18, 2022

Hi good folks, I'm new to the group, so bear with me please. A decade or more with Mm2.1 but recently upgraded to Mm3 so no experiences with this latter. Details: Debian 11.2 (stable), everything installed from deb packages, mariadb10.5, python3.9, mailman3.3.3, hyperkitty1.3.4, etc. everything stock Debian. A few low volume lists, one of them migrated from Mm2.1 but most started newly. Couple of glitches, resolved them most so far. but could not cope with this one yet so I must ask your advice. On one of my maillists certain messages can't seem to make it to the archive. They spread right among list members but stuck in /var/lib/mailman3/archives/hyperkitty/spool and hyperkitty can not handle and archive them due to finding an incorrect string. This is a mailman-web.log excerpt below believed to be relevant but please just tell me if you need further details. I didn't want to flood with longer quotation for now but I can share full Django error email if needed. 12:18:19 [Q] INFO Enqueued 8697 ERROR 2022-02-16 12:18:19,310 5898 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 71, in execute return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 226, in query _mysql.connection.query(self, query) MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x8C\\xB9\\x0D\\x0A...' for column `mailman3web`.`hyperkitty_email`.`conten t` at row 1") The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py", line 40, in inner return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py", line 81, in _decorator return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py", line 127, in archive add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py", line 153, in add_to_list email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 743, in save self.save_base(using=using, force_insert=force_insert, File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 780, in save_base updated = self._save_table( File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 873, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 910, in _do_insert return manager._insert([self], fields=fields, return_id=update_pk, File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1186, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers return executor(sql, params, many, context) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py", line 89, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 71, in execute return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 226, in query _mysql.connection.query(self, query) django.db.utils.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x8C\\xB9\\x0D\\x0A...' for column `mailman3web`.`hyperkitty_email`.`content` a t row 1") ERROR 2022-02-16 12:18:19,310 5898 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 71, in execute return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 226, in query _mysql.connection.query(self, query) MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x8C\\xB9\\x0D\\x0A...' for column `mailman3web`.`hyperkitty_email`.`conten t` at row 1") The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py", line 40, in inner return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py", line 81, in _decorator return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py", line 127, in archive add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py", line 153, in add_to_list email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 743, in save self.save_base(using=using, force_insert=force_insert, File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 780, in save_base updated = self._save_table( File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 873, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 910, in _do_insert return manager._insert([self], fields=fields, return_id=update_pk, File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1186, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers return executor(sql, params, many, context) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py", line 89, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 71, in execute return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 226, in query _mysql.connection.query(self, query) Btw. I noticed this yesterday evening and there are 5 msgs stuck this way in spool so far. Thanks for you anticipated help. Sanyi

2 7

Thanks to this very helpful community!
by Stephen Daniel Feb. 18, 2022

Feb. 18, 2022

All -- I want to express my heartfelt thanks to the people on this list who helped me stand up a modest mailman3 server. As of today I have 3 lists with a total of ~150 users running in production on a small VM in google's cloud. This implementation replaces a venerable mailman2 implementation running on servers that are to be decommissioned at some point in the near future. Thanks again! -- Stephen

1 0

FILTER_VHOST = True doesn't work fine with alias_domain
by Élodie BOSSIER Feb. 18, 2022

Feb. 18, 2022

Hello, My mailman3 installation is now functional, but the config "FILTER_VHOST = True" doesn't work fine when I define "alias_domain" for the 2 domains. I need to define that because I need to get "postfix_vmap" file. As you can see, list from others domains are visible on this URL : https://ml.franceserv.fr/postorius/lists/ I would like https://ml.franceserv.fr/postorius/lists/ and https://ml.viticreation.fr/postorius/lists/ with their own lists. Where I should check please to solve this ? Thank you in advance.

2 5

Mailman3-web/Postorius behind Lighttpd
by Élodie BOSSIER Feb. 18, 2022

Feb. 18, 2022

Hi, I'm using Mailman 3.3.3 (Tom Sawyer) on Debian Bullseye 11.2 from packages : #dpkg -l | grep mailman ii mailman3 3.3.3-1 all Mailing list management system ii mailman3-full 3.3.3-1 all Full Mailman3 mailing list management suite (metapackage) ii mailman3-web 0+20200530-2 all Django project integrating Mailman3 Postorius and HyperKitty ii python3-django-mailman3 1.3.5-2 all Django library to help interaction with Mailman3 (Python 3 version) ii python3-mailman-hyperkitty 1.1.0-10 all Mailman3 plugin to archive emails with HyperKitty ii python3-mailmanclient 3.3.2-1 all Python bindings for Mailman3 REST API (Python 3 version) I can access from localhost to http://127.0.0.1:8000/ which move me correctly to http://127.0.0.1:8000/postorius/lists/ and all seem to be correct. But 127.0.0.1:8000 listen only in localhost : #netstat -tanpu | grep 8000 tcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 626175/python3 So I try to use my lighttpd as proxy as recommanded with this config : > scgi.protocol = "uwsgi" > scgi.server = ( > "/" => (( "host" => "127.0.0.1", "port" => 8000, "check-local" => > "disable" )), > ) but it's return me a "502 Bad Gateway". I see nothing on there logs : - /var/log/lighttpd/error.log - /var/log/mailman3/web/error.log - /var/log/mailman3/web/access.log But on the file : /var/log/mailman3/web/mailman-web.log I can see that : > QUERY_STRING > REQUEST_URI/postorius/lists/REDIRECT_STATUS200 > SCRIPT_NAME" 400 - I don't know what I'm doing wrong ... There is a way to solve it with lighttpd or bind the port 8000 to 0.0.0.0 instead 127.0.0.1 (which is not recommanded) ? Thank you in advance. Best regards, Elodie

2 2

High memory usage
by Prasanth Nair Feb. 18, 2022

Feb. 18, 2022

Hi, We are having an issue with high memory usage on our Mailman3 server. Initially, our server had 16GB RAM and the memory usage went up from 5% to 92% within a week. So, we increased the memory capacity to 32GB RAM and we can see that the memory usage is going up in the same pattern. Any idea why it is happening? or any suggestions? Kind regards, Prasanth Nair

5 10

Template Ignored?
by Stephen Daniel Feb. 17, 2022

Feb. 17, 2022

Mailman3 people -- I created a new list, and used postorius to create a template for " list:user:notice:welcome". I then used the mass subscription feature to add myself to the list, having checked "pre-confirmed", "pre-approved", and "pre-verified". I received a welcome message that appears to be the default welcome message. It certainly wasn't what I put into my template. Any suggestions on how I get mailman to use my template? Things I might be doing wrong? Thanks! -- Steve

2 5

changing the main template for subscriptions and more
by Jens Günther Feb. 15, 2022

Feb. 15, 2022

Hey there, in Mailman2 there was a simple link in the moderation-emails, with which I was able to open directly the "held-messages". I know how to manually integrate such a link with the variables of mailman (https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/…) But: How can I change the template for all the lists (respectively maildomains/webdomains) so that I don't have to do this manually for all lists? I searched a bit for the corresponding template, but didn't find it. At the end it only says the following: "At your convenience, visit your dashboard to approve or deny the request." -> here I would like to add the following: "At your convenience, visit your dashboard to approve or deny the request: https://$domain/postorius/lists/$short_listname.$domain/held_messages" Or if someone sees another possibility to get us on the right track, it would be more than welcome, thank you! Kind regards, Jens.

4 7

Help with mailman user and mailman Web users
by Sherin Feb. 15, 2022

Feb. 15, 2022

Hi, I have installed mailman3 server and configured MailmanWeb ( hyperkity + Postorious ) . The service and everything works fine, but I am little bit confuses with the user management in mailman3, 1) I created an owner user for a mailing list using the mailmanclient api tools, # from mailmanclient.client import Client 2) I can't login to the Mailmanweb with this user For further investigation shows the mailman have its own database and the web frontend have its own database too. So my question is how to configure the webfrontend ( hyperkit+ postorious ) to use the accounts logins created from the mailmanclient api from the mailman core database. Or is there a way to use only one user management system ? Any help will be appreciated. -- Sherin

2 1

converting bulk accept_these_nonmembers in migration from mailman 2 to 3
by Lucio Chiappetti Feb. 12, 2022

Feb. 12, 2022

I have been instructed by Mark Sapiro to subscribe to this list to post questions related to mailman3. On Mon, 31 Jan 2022, Mark Sapiro wrote: > On 1/31/22 06:20, Lucio Chiappetti wrote: >> One of the lists of which I am a co-moderator has been moved from >> mailman 2 to 3. Actually the migration has been made operative yesterday (and so we got the first problems ...). Basic background. There are a number of lists within my institution (a nationwide research institution with many seats across our country), some are institutional (all staff subscribed ex officio) and managed by staff in one seat, some other are non-institutional (subscription is optional) and are managed by a few volunteers in other cities different from the seat where the mailman server runs. > In particular, the list settings for *_these_nonmembers are at Settings > -> Message Acceptance, but these settings in particular are deprecated > in Mailman 3 except for regexps. In Mailman 3 the preferred way to say > accept a nonmember is to create a nonmember for that user and set it's > Moderation action to Default Processing. If, a nonmember posts, the > nonmember record is automatically created, Let me explain our case, and how it was managed in mailman2. - originally our staff (of the order of 1000) had addresses like user(a)place.inaf.it or user(a)place.astro.it. Some had even two (say place1.inaf.it and place2.inaf.it), so they subscribed with ONE of their work addresses - later we got addresses like name.surname(a)inaf.it. New users subscribe with this address but MANY of the old ones continue to use the PREVIOUS address. - we implemented a spam filtering policy as follows: - staff (and former staff) can SUBSCRIBE with any address or CHANGE their address. Any subscribed address will be recognised. Mail is NOT held in moderation unless too bulky, or has undisclosed recipients - mail from non members with any address in the .it TLD or in the tng.iac.es domain (where some of our staff work) goes into moderation (this is achieved via regexp in hold_these_nonmembers) - anything else is silently discarded as spam (generic_nonmember_action = discard) - when some staff posts from a non-subscribed address (be it the home address, or one of the alternate addresses described above) it therefore goes into moderation because of hold_these_nonmembers - in mailman2 we could let them pass "manually" (if we recognised the person) or (in case they used an alternate address) tick to add them permanently to accept_these_nonmembers (in one tick) - at the end we got somehow fed up of this but we did NOT want to accept any address of the form ^.+@.+\.inaf.it (which could be forged) but found that we could simply put in accept_these_nonmembers two entries like @list1 @list2 where list1 and list2 are two institutional lists on the same server which contain all permanent and temporary staff or assimilated with their official address (these lists are automagically generated each week) (so at least any VALID EXISTING official address is accepted) What I found now (after the migration done by the administrators of the central seat) is that in mailman3: - accept_these_nonmembers is empty - there are 189 entries listed in Non-members (which look those formerly in accept_these_nonmembers) - hold_these_nonmembers was copied identical with the same regexp's as in mailman2 - I have inspected a few of them and found that they have Administration options Moderation set to List default - the effect of this apparently was that a post from some.guy(a)inaf.it (whose subscription address is guy(a)somewhere.inaf.it) was hold for moderation is this due to a loop with the regexp in hold_these_nonmembers ? the default action would be to let it pass Or should we set Administration options Moderation to Accept immediately ? - if so, how can we do it automatically for all 189 entries ? - or move back the 189 addresses to accept_these_nonmembers ? - or for the future have persons posting with alternate addrresses accepted for the future WITH ONE CLICk ? - AND LAST BUT NOT LEAST would the syntax @list1 in accept_these_nonmembers work in mailman3 ? already putting it back will save a lot of work. Many thanks -- Lucio Chiappetti - INAF/IASF - via Corti 12 - I-20133 Milano (Italy) For more info : http://www.iasf-milano.inaf.it/~lucio/personal.html ------------------------------------------------------------------------ "All that is google does not glitter Nor all who use alpine/procmail are lost"

2 3