[uWSGI] call a "streaming" LUA RPC from python

Davide Setti davide.setti at gmail.com
Wed Mar 30 11:14:50 CEST 2016


On Tue, Mar 29, 2016 at 6:38 PM, Davide Setti <davide.setti at gmail.com>
wrote:

>
> On Tue, Mar 29, 2016 at 6:32 PM, Davide Setti <davide.setti at gmail.com>
> wrote:
>
>> On Tue, Mar 29, 2016 at 5:45 PM, Riccardo Magliocchetti <
>> riccardo.magliocchetti at gmail.com> wrote:
>>
>>> One possible solution would be to write the JSON parsing in C, but I
>>> can't write
>>>
>>>> proper C code anymore... :(
>>>>
>>>
>>> Have you tried ujson? maybe it's fast enough and does not consume too
>>> much memory
>>>
>>
>> The problem is not the JSON decoder itself I guess, but keeping the whole
>> structure in memory. It's a few millions dicts, and dicts are heavy...
>> But I may be wrong, I've no numbers
>>
>
> Ok, looks like I was wrong:
> https://blog.ionelmc.ro/2015/11/22/memory-use-and-speed-of-json-parsers/
>

I did some profiling, no difference in memory and speed in my case between
the various json libs (like json vs rapidjson) except ijson that's slow but
with no memory usage, being a streaming parser.

I think I'll write a simple text parser for the JSON structure I need to
decode, in python o cython...

-- 

Davide Setti
code: http://github.com/vad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.unbit.it/pipermail/uwsgi/attachments/20160330/b841cc8c/attachment.html>


More information about the uWSGI mailing list