January 2018 - Mailman-users - lists.mailman3.org

OperationalError: database is locked
by cyp＠trojina.si Jan. 25, 2018

Jan. 25, 2018

On our installation, we are often gettting Internal Server Error (500). The logs look like this: ERROR 2018-01-09 13:52:57,640 1604 django.request Internal Server Error: /hyperkitty/list/test(a)seznami.cjvt.si/ Traceback (most recent call last): File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/exception.py", line 42, in inner response = get_response(request) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/HyperKitty-1.1.0-py2.7.egg/hyperkitty/lib/view_helpers.py", line 133, in inner mlist = MailingList.objects.get(name=mlist_fqdn) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 379, in get num = len(clone) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 238, in __len__ self._fetch_all() File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 1087, in _fetch_all self._result_cache = list(self.iterator()) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 54, in __iter__ results = compiler.execute_sql() File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/sql/compiler.py", line 835, in execute_sql cursor.execute(sql, params) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/utils.py", line 94, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/sqlite3/base.py", line 337, in execute return Database.Cursor.execute(self, query, params) OperationalError: database is locked Does this suggest I should switch from sqlite to postgres (https://stackoverflow.com/a/26864360)? I'm happy to do that, but how do I migrate the data between the databases (after installing postgres and creating the database)? I've found this doc (http://mailman.readthedocs.io/en/latest/src/mailman/docs/database.html#data…) but I don't think I understand this and I haven't used alembic: these instructions are just for when you modify the model within the same database, right? Thanks in advance!

4 9

Solved issues not in Docker-mailman rolling releases
by Henrik Rasmussen Jan. 25, 2018

Jan. 25, 2018

I am currently using Docker-mailman rolling releases and with great respect to the maintainer's time it was my impression from @maxking that rolling releases was likely to be one week behind the development. In my report of https://gitlab.com/mailman/mailman/issues/444 @msapiro commented that #444 is a duplicate of #406 (closed) which was fixed by !322 (merged), but !322 was merged a month ago. Thinking that I wasn't up to speed, I did a `docker pull quay.io/maxking/mailman-core:rolling` only got 'XXXXXXXXXXXX: Already exists'. Also `docker images` shows 'quay.io/maxking/mailman-core rolling 1654a3226db8 7 weeks ago 179.3 MB'. Not to be umpolite, did I misunderstand the concept?

2 3

Postorius: Show only own lists
by Thor Atle Rustad Jan. 23, 2018

Jan. 23, 2018

Imagine an installation with 500+ mailing lists and almost as many local admins that are supposed to maintain their own local lists with a few local members. As site admin, I see all 500+ lists, but I was expecting *local* admins to see only the lists they maintain. I have set all lists to "advertised = False", so they are hidden from ordinary view. Is there a way to show only relevant lists to local admins? I am running docker running release. mailman info GNU Mailman 3.2.0a1 (La Villa Strangiato) Thor Atle Rustad

4 6

mailing list aliases
by Dmitry Makovey Jan. 20, 2018

Jan. 20, 2018

I'm running mailman 3.3.0 and seeing odd behavior for "Settings/Message Acceptance/Acceptable aliases". Two things strike me as odd: 1. whenever I change content of that field interface keeps on wrapping it in more and more square brackets: [u'[u\'[u"[u\\\'foo(a)my.com\\\']"]\']'] whereas I've started with simple foo(a)my.com. I've checked with mailmanclient library and it seems like this is happening at the core level not just an interface: ml.settings['acceptable_aliases']=['foo(a)my.com'] ml.save() print ml.settings['acceptable_aliases'] am I using it wrong or is there a bug? 2. Related to #1 - we've migrated our mailing lists from foo(a)my.com to foo(a)lists.my.com however we've preserved old addresses. But in new setup mail sent in gets stopped for moderation with: "X-Mailman-Rule-Hits: implicit-dest" . What's the best course of action here to get mailing list respond to multiple email addresses? -- Sr System and DevOps Engineer SoM IRT

3 3

why am I seeing "Access to the archiving API endpoint was forbidden"
by Dmitry Makovey Jan. 19, 2018

Jan. 19, 2018

Running docker-based MM3 I get quite a few messages in mailmanweb.log: ERROR 2018-01-19 19:09:07,467 401 hyperkitty.views.mailman Access to the archiving API endpoint was forbidden from IP 172.19.199.1, your MAILMAN_ARCHIVER_FROM setting may be misconfigured which points to the docker host making those requests. But I have no scripts/code dunning on host to do that. I've posted to this list earlier (see "mailman-web: ResponseNotReady" and "Getting connection refused on postfix trying to connect to mailman-core:lmtp") my woes with public vs private IPs in my setup and my "hack" was to set up docker-compose with explicit domainname and hostname: mailman-core: domainname: my.com hostname: mailman ... networks: mailman: aliases: - mailman.my.com where mailman.my.com is the public IP of the service. so I think I'm in the weird state where mailman-core container starts off communication on a private docker IP and finishes it off on what it thinks is public someplace... not sure where is that coming from. I ran a cross-ping for all containers to confirm they all see only private IPs and they do: core, web & nginx cross-pinging each-other and mailman.my.com all show private docker IPs so why would any packet be routed from external? can I allow more than a single IP for MAILMAN_ARCHIVER_FROM ? ITMT I've moved MAILMAN_ARCHIVER_FROM to settings_local.py and hardwired it to be 172.199.19.1 and I believe there are no errors in log files... but again - that's a lot of tiptoeing and tapdancing around the IP assignment in docker so I'm looking for input to simplify the setup. -- Sr System and DevOps Engineer SoM IRT

1 0

MM3 & postfix ... (mail loops back to myself)
by Dmitry Makovey Jan. 18, 2018

Jan. 18, 2018

Hi, I've been running Postfix+MM3 setup for a while now. However it turns out one portion of functionality has been broken all this time: member subscription via "confirmation". Quick notes on setup: I've got 2 VMs: smtp.my.com mailman.my.com I'm running mailing lists under: lists.my.com DNS for lists.my.com A: - same as mailman.my.com MX: - smtp.my.com So when somebody sends a message to foo(a)lists.my.com they receive confirmation email (which is fine and normal). That mail suggests user needs to reply to foo-confirm+randomstuff(a)lists.my.com. Whenever user does that - postfix refuses to deliver *that* mail with reason: Action: failed Status: 5.4.6 Diagnostic-Code: X-Postfix; mail for lists.my.com loops back to myself I've got transport_maps = regexp:/opt/mailman/core/var/data/postfix_lmtp local_recipient_maps = regexp:/opt/mailman/core/var/data/postfix_lmtp relay_domains = regexp:/opt/mailman/core/var/data/postfix_domains and postfix_domains contains: /^lists\.my\.com$/ lists.my.com and postfix_lmtp contains: foo(a)lists\.my\.com$/ lmtp:[mailman.my.com]:8024 ... what am I missing? -- Sr System and DevOps Engineer SoM IRT

3 2

Hyperkitty: AttributeError: 'NoneType' object has no attribute 'subject'
by Alex McLean Jan. 14, 2018

Jan. 14, 2018

I'm getting a 500 server error with some threads in hyperkitty, e.g. this one: https://we.lurk.org/hyperkitty/list/tidal@we.lurk.org/thread/3IN5NFFFPVAQSR… The following appears in the log, any pointers much appreciated! ERROR 2018-01-13 12:29:53,855 6056 django.request Internal Server Error: /hyperkitty/list/tidal(a)we.lurk.org/thread/4ECEUJRXMJ2U7EPKRMU6V6A3FVL523BC/ Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/lib/view_helpers.py", line 136, in inner return func(request, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/views/thread.py", line 193, in thread_index return render(request, "hyperkitty/thread.html", context) File "/usr/local/lib/python2.7/dist-packages/django/shortcuts.py", line 30, in render content = loader.render_to_string(template_name, context, request, using=using) File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 68, in render_to_string return template.render(context, request) File "/usr/local/lib/python2.7/dist-packages/django/template/backends/django.py", line 66, in render return self.template.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 207, in render return self._render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 199, in _render return self.nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/loader_tags.py", line 177, in render return compiled_parent._render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 199, in _render return self.nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/loader_tags.py", line 72, in render result = block.nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/defaulttags.py", line 529, in render return self.nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/defaulttags.py", line 322, in render return nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 1040, in render output = self.filter_expression.resolve(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 708, in resolve obj = self.var.resolve(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 849, in resolve value = self._resolve_lookup(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 890, in _resolve_lookup current = getattr(current, bit) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/thread.py", line 108, in subject return self.cached_values["subject"]() File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/common.py", line 61, in __call__ return self.get_or_set(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/common.py", line 57, in get_or_set value = self.rebuild(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/common.py", line 49, in rebuild value = self.get_value(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/thread.py", line 223, in get_value return self.instance.starting_email.subject AttributeError: 'NoneType' object has no attribute 'subject' ERROR 2018-01-13 12:29:53,855 6056 django.request Internal Server Error: /hyperkitty/list/tidal(a)we.lurk.org/thread/4ECEUJRXMJ2U7EPKRMU6V6A3FVL523BC/ Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/lib/view_helpers.py", line 136, in inner return func(request, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/views/thread.py", line 193, in thread_index return render(request, "hyperkitty/thread.html", context) File "/usr/local/lib/python2.7/dist-packages/django/shortcuts.py", line 30, in render content = loader.render_to_string(template_name, context, request, using=using) File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 68, in render_to_string return template.render(context, request) File "/usr/local/lib/python2.7/dist-packages/django/template/backends/django.py", line 66, in render return self.template.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 207, in render return self._render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 199, in _render return self.nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/loader_tags.py", line 177, in render return compiled_parent._render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 199, in _render return self.nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/loader_tags.py", line 72, in render result = block.nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/defaulttags.py", line 529, in render return self.nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/defaulttags.py", line 322, in render return nodelist.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 1040, in render output = self.filter_expression.resolve(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 708, in resolve obj = self.var.resolve(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 849, in resolve value = self._resolve_lookup(context) File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 890, in _resolve_lookup current = getattr(current, bit) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/thread.py", line 108, in subject return self.cached_values["subject"]() File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/common.py", line 61, in __call__ return self.get_or_set(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/common.py", line 57, in get_or_set value = self.rebuild(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/common.py", line 49, in rebuild value = self.get_value(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/hyperkitty/models/thread.py", line 223, in get_value return self.instance.starting_email.subject AttributeError: 'NoneType' object has no attribute 'subject'

2 3

Django logins for new subscribers
by Alex McLean Jan. 13, 2018

Jan. 13, 2018

Hi all, I'm moving our groups to mailman3 (http://we.lurk.org) Enjoying it so far! I've mass-subscribed people via postorius. However the subscribers don't have django (postorius/hyperkitty) logins. It seems they only exist on the mailman side, not the django side. If they go through the process of creating an account in postorius, they can configure their mailing list subscriptions, but this is a bit fiddly. Is this expected behaviour? It would be great if they could be sent a temporary link to set a password and go. Best wishes, alex

3 4

hyperkitty connection issues
by Dmitry Makovey Jan. 11, 2018

Jan. 11, 2018

I'm using docker-based deployment of mailman. things appear to be functioning for the most part, but recently I've started getting reports from users stating that mail from *some* subscribers to the list gets queued for moderation with reason "n/a". Not sure if that's related, but I've also noticed that my logs are filling up with Connection to HyperKitty failed: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) (full dump: https://gist.github.com/droopy4096/1538d34ed7b6709252e8c3e82aaca69d ) what's the best angle to troubleshoot this situation? Note: I have restarted containers within the last 24 hours and still am observing above behavior (both mail issues and connection issues in logs) -- Sr System and DevOps Engineer SoM IRT

2 1

Hyperkitty import from mailman2
by Sten Aus Jan. 3, 2018

Jan. 3, 2018

Hi I have installed mailman3 - it looks very great! Now I wanted to try import from mailman2 mbox, so i did: python2 manage.py hyperkitty_import --settings settings --ignore-mtime -v 3 --since 2010-01-01 -l test(a)list.domain.com /tmp/test.mbox It shows that it have sucessfully imported all emails and I can see them from database (sqlite3), but I cannot see them from GUI (hyperkitty). Only thing I can see is this inactive "test" list. I have tried that I create list before importing, still same result. All new emails are archiving to another lists (which are not imported, but new installations). Any help is appreciated! All the best Sten

4 8