Parquet to JSON: Fix "Object Is Not JSON Serializable"

The error

You read a Parquet file, turn it into records and dump it:

import json, pandas as pd

df = pd.read_parquet("orders.parquet")
json.dumps(df.to_dict(orient="records"))
# TypeError: Object of type Timestamp is not JSON serializable

Parquet is strongly typed and JSON is not. Python's json module only encodes dict, list, str, int, float, bool and None. Everything pandas and pyarrow hand you out of a typed Parquet column is something else.

Which Parquet types break it

The last row is the confusing one: to_dict() usually converts numpy scalars back to Python ints, but values pulled out with .iloc, .values or inside list columns often stay as numpy.int64, which json refuses.

Fix 1: let pandas do the encoding

pandas has its own encoder that knows these types:

df.to_json("orders.json", orient="records", date_format="iso")
# or one object per line
df.to_json("orders.jsonl", orient="records", lines=True, date_format="iso")

Pass date_format="iso" explicitly. With orient="records" the default is epoch, which writes timestamps as millisecond integers — valid JSON, but rarely what the consumer expects. to_json also writes missing values as null instead of the invalid NaN token that json.dumps produces.

Fix 2: a default= hook that makes deliberate choices

If you need json.dumps (to post to an API, say), decide per type instead of reaching for default=str:

import base64, datetime, decimal
import numpy as np

def encode(obj):
    if isinstance(obj, (datetime.datetime, datetime.date)):
        return obj.isoformat()
    if isinstance(obj, decimal.Decimal):
        return str(obj)          # keep exact precision; use float(obj) if lossy is OK
    if isinstance(obj, bytes):
        return base64.b64encode(obj).decode("ascii")
    if isinstance(obj, np.ndarray):
        return obj.tolist()
    if isinstance(obj, np.generic):
        return obj.item()
    raise TypeError(f"Unhandled type: {type(obj).__name__}")

records = df.astype(object).where(df.notna(), None).to_dict(orient="records")
payload = json.dumps(records, default=encode)

pandas.Timestamp subclasses datetime.datetime, so the first branch covers it. The where(df.notna(), None) step replaces NaN and NaT with None, so the output contains null. Raising on unknown types is on purpose: a new column type should fail loudly, not turn into a string nobody asked for.

Fix 3: skip Python and export with DuckDB

DuckDB maps every Parquet type to JSON itself:

COPY (SELECT * FROM 'orders.parquet')
TO 'orders.json' (FORMAT JSON, ARRAY true);

Timestamps come out as ISO strings, lists as JSON arrays, structs as nested objects and nulls as null. To control the representation of a column, cast it in the SELECT: CAST(amount AS VARCHAR) keeps a DECIMAL exact, and strftime(created_at, '%Y-%m-%dT%H:%M:%SZ') fixes the timestamp format.

You can run the same statement in the SQL Workbench, or use the Parquet to JSON converter (or Parquet to JSONL for line-delimited output) to convert the whole file in your browser without writing any code.

Check the output

Before handing the file to someone else, check that a strict parser accepts it:

python -c "import json,sys; json.load(open(sys.argv[1]))" orders.json
jq length orders.json

If either one complains about NaN or Infinity, some float column still has non-finite values. Find them with SELECT count(*) FROM 'orders.parquet' WHERE isnan(price) OR isinf(price) and decide whether they should be null or dropped.