[uWSGI] call a "streaming" LUA RPC from python

Davide Setti davide.setti at gmail.com
Tue Mar 29 18:32:03 CEST 2016


Thank you Riccardo,

On Tue, Mar 29, 2016 at 5:45 PM, Riccardo Magliocchetti <
riccardo.magliocchetti at gmail.com> wrote:

> One possible solution would be to write the JSON parsing in C, but I can't
> write
>
>> proper C code anymore... :(
>>
>
> Have you tried ujson? maybe it's fast enough and does not consume too much
> memory
>

The problem is not the JSON decoder itself I guess, but keeping the whole
structure in memory. It's a few millions dicts, and dicts are heavy... But
I may be wrong, I've no numbers

Another solution would be to call a LUA RPC, but we would have memory
>> problems
>> because uWSGI RPC only supports strings as an exchange format, a single
>> string I
>> think. Are there tricks to easily dump to file/unix pipes or something?
>>
>
> If you want to go down this route maybe you can send to lua just the query
> url, process the file in lua and just return the processed output position
> instead of its full content?
>

Any experience spawning something like a thread in LUA, returning before it
finishes and consuming the generated file from python? In this way I would
be able to speed up response writing.

Thank you again.
-- 

Davide Setti
code: http://github.com/vad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.unbit.it/pipermail/uwsgi/attachments/20160329/09587f7e/attachment.html>


More information about the uWSGI mailing list