very slow performance with global sync: True setting vs. async(sync=True)

[elnygren]

The async function is very slow with Q_CLUSTER['sync']=True if the sync=True parameter is not given to async.

I have some testcases in my own application:

class ApplicationTestCase(TestCase):
    def test_async_slow(self):
        from time import time
        start = time()

        self.assertEqual(settings.Q_CLUSTER['sync'], True)

        for i in range(0, 5):
            async('django.core.mail.send_mail',
                    subject        = 'Invitation to join team {} at BuzzCo'.format('asd'),
                    from_email     = settings.DEFAULT_FROM_EMAIL,
                    message        = 'hi',
                    recipient_list = ['someone@corporate.com'],
                    fail_silently  = False,
            )
            print(time() - start)
        self.assertEqual(True, False)   #fail to get output

    def test_async_fast(self):
        from time import time
        start = time()

        self.assertEqual(settings.Q_CLUSTER['sync'], True)

        for i in range(0, 5):
            async('django.core.mail.send_mail',
                    subject        = 'Invitation to join team {} at BuzzCo'.format('asd'),
                    from_email     = settings.DEFAULT_FROM_EMAIL,
                    message        = 'hi',
                    recipient_list = ['someone@corporate.com'],
                    fail_silently  = False,
                    async=True
            )
            print(time() - start)
        self.assertEqual(True, False)   #fail to get output

outputs:

test_async_fast
0.021257877349853516
0.02476787567138672
0.027889013290405273
0.030992984771728516
0.0339360237121582

test_async_slow
5.010109901428223
5.0143818855285645
5.018315076828003
5.022057056427002
5.025748014450073

Notes:

• Django==1.9.7
• django-q==0.7.18
• q-cluster broker is sqlite ('orm')
• emailbackend is the CONSOLE backend

[elnygren]

Noticed that

async=True

should be actually be

sync=True

which probably means that test_async_fast was fast because it failed immediately.

Doesn't yet explain why slow took 5s per test though.

[elnygren]

after some profiling I've found the reason to be python's socket.getfqdn() taking over 5s so nothing related to Django-Q. Closing the issue.

Solution for those are interested: I replaced Django's emailbackend during tests to a custom one that does not call that function.

class TestEmailBackend(BaseEmailBackend):
    def send_messages(self, messages):
        mail.outbox.extend(messages)
        return len(messages)