Skip to content

pyroid.json_repair

A drop-in, Rust-backed replacement for json_repair 0.64.5 (import pyroid.json_repair).

repair_json, loads, load and from_file take json_repair's arguments and return the same strings, objects and repair logs. Valid JSON comes back as json.dumps(json.loads(text)). Broken JSON (LLM output with missing quotes, single quotes, comments, trailing commas, truncation, markdown fences, prose around it, Python literals, ...) is repaired with json_repair's heuristics, ported line by line to Rust. The port stays linear on 13 input families where json_repair's lookaheads are quadratic, and never overflows the stack.

These are delegated to the real json_repair, with a one-time pyroid.FallbackWarning: every schema= call, non-str input, and the rare text holding a char in U+10F800..U+10FFFF (the range the port uses to store lone surrogates). Without json_repair installed they raise pyroid.PyroidError.

Differences from json_repair

  • json_repair raises ValueError past about 140-330 levels of broken input (it depends on sys.getrecursionlimit() and the caller's stack). pyroid repairs up to MAX_DEPTH levels, and deeper broken input raises NestingError, which is both a ValueError and a RecursionError.
  • Valid JSON is decoded by json.loads for objects, as in json_repair, so its RecursionError past about 10,000 levels on 3.12 is the same. For string results pyroid decodes it at any depth (json_repair raises there too).
  • load/from_file read the file once (fd.read()) instead of in chunk_length pieces. The result is the same, chunk_length is ignored and unseekable streams work.
  • json_repair's command line (python -m json_repair) is not part of this module.

JSON_REPAIR_VERSION module-attribute 

JSON_REPAIR_VERSION: Final[str]

DELEGATED module-attribute 

DELEGATED: Final[tuple[str, ...]]

MAX_DEPTH module-attribute 

MAX_DEPTH: Final[int]

JSONReturnType module-attribute 

JSONReturnType: TypeAlias = (
    dict[str, Any]
    | list[Any]
    | str
    | float
    | int
    | bool
    | None
)

NestingError

Bases: PyroidError, ValueError, RecursionError

The input nests deeper than MAX_DEPTH levels.

repair_json 

repair_json(
    json_str: str = "",
    return_objects: Literal[False] = False,
    skip_json_loads: bool = False,
    logging: Literal[False] = False,
    json_fd: TextIO | None = None,
    chunk_length: int = 0,
    stream_stable: bool = False,
    strict: bool = False,
    schema: Any | None = None,
    schema_repair_mode: Literal[
        "standard", "salvage"
    ] = "standard",
    **json_dumps_args: Any,
) -> str

repair_json(
    json_str: str = "",
    return_objects: Literal[True] = True,
    skip_json_loads: bool = False,
    logging: Literal[False] = False,
    json_fd: TextIO | None = None,
    chunk_length: int = 0,
    stream_stable: bool = False,
    strict: bool = False,
    schema: Any | None = None,
    schema_repair_mode: Literal[
        "standard", "salvage"
    ] = "standard",
    **json_dumps_args: Any,
) -> JSONReturnType

repair_json(
    json_str: str = "",
    return_objects: bool = False,
    skip_json_loads: bool = False,
    logging: Literal[True] = True,
    json_fd: TextIO | None = None,
    chunk_length: int = 0,
    stream_stable: bool = False,
    strict: bool = False,
    schema: Any | None = None,
    schema_repair_mode: Literal[
        "standard", "salvage"
    ] = "standard",
    **json_dumps_args: Any,
) -> tuple[JSONReturnType, list[dict[str, str]]]

repair_json(
    json_str: str = "",
    return_objects: bool = False,
    skip_json_loads: bool = False,
    logging: bool = False,
    json_fd: TextIO | None = None,
    chunk_length: int = 0,
    stream_stable: bool = False,
    strict: bool = False,
    schema: Any | None = None,
    schema_repair_mode: Literal[
        "standard", "salvage"
    ] = "standard",
    **json_dumps_args: Any,
) -> (
    str
    | JSONReturnType
    | tuple[JSONReturnType, list[dict[str, str]]]
)

Repair a JSON string, like json_repair.repair_json.

Parameters:

  • json_str (str, default: '' ) –

    The text to repair.

  • return_objects (Literal[False] | Literal[True] | bool, default: False ) –

    Return the decoded value instead of a JSON string.

  • skip_json_loads (bool, default: False ) –

    Skip the json.loads validity check (valid JSON then goes through the repair parser too, as in json_repair).

  • logging (Literal[False] | Literal[True] | bool, default: False ) –

    Return (value, repair_log); the log is a list of {"text": ..., "context": ...} dicts, empty when nothing was repaired.

  • json_fd (TextIO | None, default: None ) –

    Read the text from this file instead (prefer load).

  • chunk_length (int, default: 0 ) –

    Ignored (json_repair's file chunk size).

  • stream_stable (bool, default: False ) –

    Keep repairs of a growing streamed prefix stable.

  • strict (bool, default: False ) –

    Raise ValueError on structural problems instead of repairing them.

  • schema (Any | None, default: None ) –

    A JSON Schema or pydantic model guiding the repair (delegated to json_repair; needs json_repair, and jsonschema/pydantic as it does).

  • schema_repair_mode (Literal['standard', 'salvage'], default: 'standard' ) –

    "standard" or "salvage" (requires schema).

  • **json_dumps_args (Any, default: {} ) –

    Passed to json.dumps for the string result (ensure_ascii, indent, separators, sort_keys, allow_nan are reproduced in Rust; others call json.dumps).

Returns:

  • str | JSONReturnType | tuple[JSONReturnType, list[dict[str, str]]] | str | JSONReturnType | tuple[JSONReturnType, list[dict[str, str]]] –

    The repaired JSON string ("" when nothing was found), the value with return_objects, or (value, log) with logging.

Raises:

  • ValueError –

    In strict mode, for bad schema_repair_mode combinations, or from int() limits, as json_repair does.

  • NestingError –

    The input nests deeper than MAX_DEPTH.

loads 

loads(
    json_str: str,
    skip_json_loads: bool = False,
    logging: bool = False,
    stream_stable: bool = False,
    strict: bool = False,
    schema: Any | None = None,
    schema_repair_mode: Literal[
        "standard", "salvage"
    ] = "standard",
) -> (
    JSONReturnType
    | tuple[JSONReturnType, list[dict[str, str]]]
    | str
)

Like json.loads, but repairs the text first (json_repair.loads).

load 

load(
    fd: TextIO,
    skip_json_loads: bool = False,
    logging: bool = False,
    chunk_length: int = 0,
    strict: bool = False,
    schema: Any | None = None,
    schema_repair_mode: Literal[
        "standard", "salvage"
    ] = "standard",
) -> (
    JSONReturnType
    | tuple[JSONReturnType, list[dict[str, str]]]
)

Like json.load, but repairs the text first (json_repair.load).

from_file 

from_file(
    filename: str | Path,
    skip_json_loads: bool = False,
    logging: bool = False,
    chunk_length: int = 0,
    strict: bool = False,
    schema: Any | None = None,
    schema_repair_mode: Literal[
        "standard", "salvage"
    ] = "standard",
) -> (
    JSONReturnType
    | tuple[JSONReturnType, list[dict[str, str]]]
)

Read and repair the JSON in filename (json_repair.from_file).

repair_json_batch 

repair_json_batch(
    texts: str,
    *,
    threads: int | None = None,
    skip_json_loads: bool = False,
    stream_stable: bool = False,
    strict: bool = False,
    **json_dumps_args: Any,
) -> str

repair_json_batch(
    texts: list[str],
    *,
    threads: int | None = None,
    skip_json_loads: bool = False,
    stream_stable: bool = False,
    strict: bool = False,
    **json_dumps_args: Any,
) -> list[str]

repair_json_batch(
    texts: tuple[str, ...],
    *,
    threads: int | None = None,
    skip_json_loads: bool = False,
    stream_stable: bool = False,
    strict: bool = False,
    **json_dumps_args: Any,
) -> tuple[str, ...]

repair_json_batch(
    texts: ArrowStrings,
    *,
    threads: int | None = None,
    skip_json_loads: bool = False,
    stream_stable: bool = False,
    strict: bool = False,
    **json_dumps_args: Any,
) -> Column[str]

repair_json over many documents at once, on pyroid's thread pool.

Not part of json_repair. Each result equals json_repair.repair_json(doc, skip_json_loads=..., stream_stable=..., strict=..., **json_dumps_args); a document whose repair equals its input comes back as the original object.

Parameters:

  • texts (str | list[str] | tuple[str, ...] | ArrowStrings) –

    A str, a list or tuple of str, or an Arrow string array/column (string, large_string or string_view; nulls stay null).

  • threads (int | None, default: None ) –

    None uses pyroid's shared thread pool; n runs at most n tasks at once; 1 runs on the calling thread.

  • skip_json_loads (bool, default: False ) –

    As for repair_json.

  • stream_stable (bool, default: False ) –

    As for repair_json.

  • strict (bool, default: False ) –

    As for repair_json.

  • **json_dumps_args (Any, default: {} ) –

    As for repair_json.

Returns:

  • str | list[str] | tuple[str, ...] | Column[str] –

    The same kind as texts: str, list, tuple, or a pyroid.Column of strings (large_string for large_string input, else string).

Raises: