[uWSGI] Automatically recovering from uWSGI listen queue full

Barthelemy Dagenais bart at resulto.ca
Wed Sep 26 10:25:16 UTC 2018


Hi, we are using uWSGI in emperor mode on AWS and we recently encountered a
bad case of EBS drive degradation that forced us to restart uWSGI once the
hard drive was working correctly again. My main question is: is there any
configuration option that would have allowed us to automatically restart
uWSGI.

The details:

1. We use uWSGI 2.0.17 in emperor mode and nginx is used as a reverse proxy.

2. Our main application has the following configuration:
```
[uwsgi]
socket = 127.0.0.1:8999
pythonpath = /path/to/app
virtualenv = /path/to/virtualenv
processes = 6
max-requests = 100
reload-on-rss = 100
master = true
harakiri = 20
module = our_app.wsgi
pidfile = /var/run/webapp/our_app.pid
post-buffering = 4096
logger = syslog:uwsgi.our_app
disable-logging = true
log-date = true
log-slow = 1000
log-5xx = true
log-maxsize = 16777216
```

3. When the EBS drive started behaving erratically, we started receiving
hundreds of harakiri notifications:

`Sep 25 20:10:15 our_server uwsgi.our_app: Tue Sep 25 20:05:37 2018 - ***
HARAKIRI ON WORKER 1 (pid: 3720, try: 382) **`

4. Then we started to see this log statement:

`Sep 25 20:13:35 our_server uwsgi.our_app: Tue Sep 25 20:08:57 2018 - ***
uWSGI listen queue of socket "127.0.0.1:8999" (fd: 6) full !!! (101/100)
***`

5. Once the hard drive recovered, new requests were still returning 504
gateway timeout error for several minutes. Restarting uWSGI "fixed" the
problem.

My uninformed guess is that uWSGI was still trying to process the listen
queue, but I wonder if there is some configuration parameter or some
statistics emitted by uWSGI that we could use to automate a restart if this
ever happens again.

In any case, thanks for this wonderful piece of software!

-- 
*Barthélémy Dagenais*
Chef de la technologie | Chief Technology Officer

t. 1 (877) 672-2844 x202
m. (438) 324-0860
Resulto.ca <http://www.resulto.ca> · Linkedin
<https://www.linkedin.com/company/resulto>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.unbit.it/pipermail/uwsgi/attachments/20180926/cbff2f74/attachment.html>


More information about the uWSGI mailing list