March 2018 - Mailman-users - lists.mailman3.org

Templates and docker-mailman
by jwhathaway＠locklaw.com March 20, 2018

March 20, 2018

I have mailman installed through docker-mailman and I'm trying to get rid of the footer decoration in our messages. Based on earlier threads, I have placed an empty file on the host running Docker at: /opt/mailman/core/var/templates/site/en/list:member:generic:footer But this is not being honored by the mailman container. I have restarted all of the containers. Is there something else I need to do?

3 3

Re: Getting rid of held messages from the command line
by Mark Sapiro March 19, 2018

March 19, 2018

On 03/19/2018 02:22 PM, Aymeric Mansoux wrote: > > Ah, so if I understand you correctly, as long as I'm on 3.1.1, I could > run into this issue again, but then: Yes, this issue will recur with Mailman 3.1.1 and a MySQL database. > - Would patching 3.1.1 with 5477a201 be enough to prevent further issues > until the next stable is released? I think so. The only potential issue is if mailman/database/alembic/versions/4bd95c99b2e7_fix_template_password.py is not in 3.1.1. I think it is, so I think it's OK, but if not, you'd need to modify mailman/database/alembic/versions/3f31035ed0d7_further_increase_pendedkeyvalue_value_.py to point to the 3.1.1 head of the alembic versions. > - How can I clean the current pendedkeyvalue table? With phpMyAdmin or the mysql command. I think you can just delete those entries from the pendedkeyvalue table. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Getting rid of held messages from the command line
by Aymeric Mansoux March 17, 2018

March 17, 2018

Hello, We're currently running some of Mailman software, as available via pip, namely: - mailman 3.1.1 (pip3) - postorius 1.1.2 (pip2) - hyperkitty 1.1.4 (pip2) - mailmanclient 3.1.1 (pip3) So far so good, however, for some reasons, a couple of our lists have started to accumulate held messages that cannot be manipulated via the REST API. Regardless if we use the web interface or mailmanclient, it ends up with: Mar 14 10:58:32 2018 (24287) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3.5/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "/usr/local/lib/python3.5/dist-packages/falcon/api.py", line 244, in __call__ responder(req, resp, **params) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 167, in on_get resource = self._make_collection(request) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/local/lib/python3.5/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 157, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 70, in _make_resource resource = super()._make_resource(request_id) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 39, in _make_resource results = requests.get_request(request_id) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/model/requests.py", line 120, in get_request result.data_hash, expunge=False) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/model/pending.py", line 138, in confirm value = json.loads(keyvalue.value) File "/usr/lib/python3.5/json/__init__.py", line 319, in loads return _default_decoder.decode(s) File "/usr/lib/python3.5/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.5/json/decoder.py", line 355, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1 (char 0) Mar 14 10:58:32 2018 (24287) 127.0.0.1 - - "GET /3.1/lists/xx(a)xx.xx/held HTTP/1.1" 500 59 Is there a way to manipulate/see/remove these mails differently? Right now this prevent moderation of any held messages (note: some lists do not suffer from this and the held panel or mailmanclient works perfectly fine). Thanks!

3 4

disable sign-ups
by Peter Münster March 14, 2018

March 14, 2018

Hi, I would like to add new accounts only manually, the mailman3 site should not be open for sign-ups. So I've added this code to settings.py: --8<---------------cut here---------------start------------->8--- from allauth.account.adapter import DefaultAccountAdapter class MyAccountAdapter(DefaultAccountAdapter): def is_open_for_signup(self, request): return False ACCOUNT_ADAPTER = 'MyAccountAdapter' --8<---------------cut here---------------end--------------->8--- It seems to work, but the user experience is not optimal: the user can click on "Sign Up", and then gets to "Server Error (500)". This is the log: --8<---------------cut here---------------start------------->8--- ERROR 2018-01-31 21:53:47,116 6769 django.request Internal Server Error: /accounts/signup/ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 34, in _wrapper return bound_func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper return view(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 30, in bound_func return func.__get__(self, type(self))(*args2, **kwargs2) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 210, in dispatch return super(SignupView, self).dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 79, in dispatch **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 182, in dispatch if not self.is_open(): File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 191, in is_open return get_adapter(self.request).is_open_for_signup(self.request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/adapter.py", line 501, in get_adapter return import_attribute(app_settings.ADAPTER)(request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/utils.py", line 150, in import_attribute pkg, attr = path.rsplit('.', 1) ValueError: need more than 1 value to unpack ERROR 2018-01-31 21:53:47,116 6769 django.request Internal Server Error: /accounts/signup/ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 34, in _wrapper return bound_func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper return view(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 30, in bound_func return func.__get__(self, type(self))(*args2, **kwargs2) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 210, in dispatch return super(SignupView, self).dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 79, in dispatch **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 182, in dispatch if not self.is_open(): File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 191, in is_open return get_adapter(self.request).is_open_for_signup(self.request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/adapter.py", line 501, in get_adapter return import_attribute(app_settings.ADAPTER)(request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/utils.py", line 150, in import_attribute pkg, attr = path.rsplit('.', 1) ValueError: need more than 1 value to unpack --8<---------------cut here---------------end--------------->8--- Do you have any hints, how to avoid such errors please? TIA for any help, -- Peter

3 2

Internal Server Error: /hyperkitty/api/mailman/archive: ValueError: A string literal cannot contain NUL (0x00) characters.
by Henrik Rasmussen March 14, 2018

March 14, 2018

This could potentially be associated with [1] and [2], but it does not regard import21. When mails are released or otherwise archived I get an Internal Server Error regarding "ValueError: A string literal cannot contain NUL (0x00) characters." in the web/mailmanweb.log, but unlike importing corrupted archives, this is mails coming from the users in the daily life. Would it be possible for Mailman to check for NUL characters and skip those before archiving? ERROR 2018-03-13 10:38:12,602 39 django.request Internal Server Error: /hyperkitty/api/mailman/archive Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python2.7/site-packages/hyperkitty/views/mailman.py", line 68, in _decorator return func(request, *args, **kwargs) File "/usr/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view return view_func(*args, **kwargs) File "/usr/local/lib/python2.7/site-packages/hyperkitty/views/mailman.py", line 114, in archive add_to_list(mlist_fqdn, msg) File "/usr/local/lib/python2.7/site-packages/hyperkitty/lib/incoming.py", line 149, in add_to_list email.save() File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 796, in save force_update=force_update, update_fields=update_fields) File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 824, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 908, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 947, in _do_insert using=using, raw=raw) File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 1043, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1054, in execute_sql cursor.execute(sql, params) File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) ValueError: A string literal cannot contain NUL (0x00) characters. [1] https://gitlab.com/mailman/hyperkitty/issues/155 [2] https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/….

1 1

A short link to this page -- TinyURL
by John Griessen March 14, 2018

March 14, 2018

The wiki page has a convenient TinyURL. Is it feasible to make TinyURLs for archived list threads such as: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…

3 2

Database running VERY hot
by Darren Smith March 12, 2018

March 12, 2018

Hello all, We had an old installation of Mailman 2.1 across three servers with over 32,000 mailing lists on them. Due to circumstances beyond my control, we had to shut them down and quickly set up Mailman 3 on a new server. We have Mailman3 set up on a single server this time, as we were having problems with a 3-server setup and Hyperkitty and archives. Long story short, it is up and running, and our users are starting to come back. However, we do not have very many users hitting Postorius and Hyperkitty yet, and we are seeing a distressing number of 504 gateway timeouts. We noticed that there is a significant number of queries to the API whenever hitting Postorius's front page to load the mailing lists, and then on every load. It looks as if it makes a single call to the API to get information about a single list, and does this 10 times - once for each list - to render the page. This gets much worse when users set the view to 200 per page, as they are inclined to do with 32,000 mailing lists... What we've done, actually, is to override the front page and point the users to a different set of static pages where we have all of our lists categorized and searchable, and when they find the list they are looking for, we have a link back to postorius. It has taken a lot of the stress off of the server. However, I still see where the database at times can run VERY hot. We have set up this server to run with PostgreSQL. When I do a ps aux | grep postgres, I find a lot of processes that have been spawned to take care of mailman - a dozen or so. However, one of the threads is taking up about 85% of the load, another thread about 14%, and little dribbles here and there from the rest of the threads. Here are the questions: 1) Has anyone run into problems like these? I would think that maybe this is an issue having so many mailing lists, but honestly there aren't that many people using the system yet. It is going to ramp up as the users realize that we are live again, but we aren't getting that much traffic. 2) Is there any configuration that I can set, especially with regards to the database, so that it can be used more efficiently and not throttle through one or two threads? Please let me know what further information I can provide. Thanks in advance for any help or advice! Silas

5 11

Deactivate list without deleting it?
by Colin Watson March 3, 2018

March 3, 2018

Hi, I'm experimenting with a port of Launchpad's mailing list infrastructure to Mailman 3. If Barry's reading, he may remember that this is no trivial task, so I'm very definitely not promising that it will be done any time soon. :-) However, for the most part it looks relatively tractable with the aid of the plugin improvements in Mailman 3.2.0; I have the rough outline of a plugin that implements much the same feature set as our current pile of monkey-patches against Mailman 2.1 in a much nicer way. One of the major things I haven't figured out yet is list deactivation and reactivation. In our current 2.1-based code, you can deactivate a list in a reversible way, which basically tars up the list directory so that the list archives can be restored later. This isn't a native operation in 2.1, but it could be done because everything lived on the filesystem. In 3, it's less trivial. We could go to some effort to dump things out of the HyperKitty database and restore it on reactivation, but that seems rather complex and a bit silly. Ideally, I'd like mailing lists to have a "deactivated" state in addition to full deletion. In this state, the list would appear nonexistent for most purposes: incoming email would be rejected, Postorius would deny its existence, etc. There'd need to be corresponding extensions to the IArchiver interface which caused the archiver to put its archives of that list into a corresponding deactivated state, in which it would retain the messages but queries for them would return 404 or similar; these extensions would need to be optional, since while it should be relatively easy to do this in HyperKitty it could be hard or impossible for other archivers, and perhaps this should be a separate operation from deactivating the list itself since people may want to close a list while leaving its archives accessible. Has there been any discussion of this in the past, or any advice as to implementation strategy? Thanks, -- Colin Watson [cjwatson(a)ubuntu.com]

3 7