Skip to content
This repository has been archived by the owner on Mar 8, 2018. It is now read-only.

Update iainb's pull request to merge cleanly + socket support + decode a buffer at a time + bugfixes #38

Open
wants to merge 25 commits into
base: master
Choose a base branch
from

Conversation

otherwiseguy
Copy link

This pull request:

  1. Updates iainb's pull request(Iterate over a stream of json objects + support new yajl version + bug fixes #33) which no longer applies cleanly
  2. Adds the ability to use a socket as a stream by supporting recv() if read() is not available
  3. Adds the ability to create a decoder and then decode strings iteratively. This could be useful if you are reading a buffer of characters at a time from a non-blocking socket. A contrived example:
import yajl

chars = list("""[1,2,3,{"key": "value"}]{"number": 1}[null]""")

d = yajl.Decoder(allow_multiple_values=True)

for c in chars:
    results = d.iterdecode(c)
    for r in results:
        print r
    else:
        print "Reading..."
  1. In ianb's iterator patch, fixed the refcount on passing in bufsize which could cause a segfault.

iainb and others added 25 commits January 24, 2015 21:59
* updated build instructions
rickeyski@cf3c29d

Move IssueTwentySevenTest dict into python2 to prevent python3 test from bailing on loading the test suite

Update IssueTwentySevenTest to only run on python2. (probably want to run this on python3 too).
* Store decoded objects in a list
Remove actions from _internal_decode that reset the parser state
clang -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I. -Iincludes/ -Iyajl/src -I/usr/include/python2.6 -c yajl.c -o build/temp.linux-x86_64-2.6/yajl.o -Wall -DMOD_VERSION="0.3.6-38862b0"
yajl.c:411:15: warning: incompatible pointer types initializing 'PyCFunction' (aka 'PyObject *(*)(PyObject *, PyObject *)') with
      an expression of type 'PyCFunctionWithKeywords' (aka 'PyObject *(*)(PyObject *, PyObject *, PyObject *)')
    {"dumps", (PyCFunctionWithKeywords)(py_dumps), METH_VARARGS | METH_KEYWORDS,
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
yajl.c:427:14: warning: incompatible pointer types initializing 'PyCFunction' (aka 'PyObject *(*)(PyObject *, PyObject *)') with
      an expression of type 'PyCFunctionWithKeywords' (aka 'PyObject *(*)(PyObject *, PyObject *, PyObject *)')
    {"dump", (PyCFunctionWithKeywords)(py_dump), METH_VARARGS | METH_KEYWORDS,
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
yajl.c:331:18: warning: unused function 'py_iterload' [-Wunused-function]
static PyObject *py_iterload(PYARGS)
                 ^
3 warnings generated.

clang -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I. -Iincludes/ -Iyajl/src -I/usr/include/python2.6 -c encoder.c -o build/temp.linux-x86_64-2.6/encoder.o -Wall -DMOD_VERSION="0.3.6-38862b0"
encoder.c:316:9: warning: expression result unused [-Wunused-value]
        PyObject_INIT_VAR(op, &PyString_Type, size);
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from encoder.c:32:
In file included from /usr/include/python2.6/Python.h:81:
/usr/include/python2.6/objimpl.h:159:29: note: instantiated from:
    ( Py_SIZE(op) = (size), PyObject_INIT((op), (typeobj)) )
                            ^
encoder.c:316:9: note: instantiated from:
        PyObject_INIT_VAR(op, &PyString_Type, size);
        ^                 ~~
encoder.c:316:27: note: instantiated from:
        PyObject_INIT_VAR(op, &PyString_Type, size);
                          ^~
1 warning generated.
Updated runtests.sh to run issue_11 test under python3
To iterate over json items written to the stdin of a python process:

import yajl
import sys

for i in yajl.Decoder(allow_multiple_values=True,stream=sys.stdin):
    print i

Python3 support is broken in this build.
Refactor py_yajldecoder_decode
add check to decoder init function to check stream has read() method
Also fix functions which were returning true/false/none types but not incrementing the refcount
Unicode buffer was not being freed after use
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants