Convert a JSON Array File to Parquet (Not JSONL)

The situation

An API dump or export tool gave you events.json that looks like this:

[
  {"id": 1, "user": {"name": "Ana", "plan": "pro"}, "tags": ["a", "b"], "ts": "2026-10-01T09:12:00Z"},
  {"id": 2, "user": {"name": "Ben", "plan": "free"}, "tags": [], "ts": "2026-10-01T09:15:30Z"}
]

It is not JSONL — it is one array — and you want Parquet so it is smaller, typed and queryable. DuckDB handles this without pandas.

Plain array of objects

COPY (
  SELECT * FROM read_json('events.json', format = 'array')
) TO 'events.parquet' (FORMAT parquet, COMPRESSION zstd);

read_json samples the records and infers a schema: id becomes BIGINT, ts becomes TIMESTAMP, user becomes a STRUCT(name VARCHAR, plan VARCHAR) and tags a VARCHAR[]. Those nested types are written as native Parquet structs and lists, not as JSON strings. Check what was inferred before writing:

DESCRIBE SELECT * FROM read_json('events.json', format = 'array');

As a one-liner from a shell:

duckdb -c "COPY (FROM read_json('events.json', format='array')) TO 'events.parquet' (FORMAT parquet)"

Records wrapped in an object

Many APIs return {"count": 2, "data": [ ... ]}. Read the outer object and unnest the array:

COPY (
  SELECT unnest(data, recursive := true)
  FROM read_json('response.json', maximum_object_size = 200000000)
) TO 'records.parquet' (FORMAT parquet);

recursive := true expands each record's fields into columns. Drop it if you prefer one struct column per record.

Errors you will hit

• "maximum_object_size of 16777216 bytes exceeded" — the whole file is one JSON value (common in the wrapped case). Set maximum_object_size to more than the file size in bytes, as above. For multi-GB arrays, converting to JSONL once with jq -c '.[]' events.json > events.jsonl is faster and streams.

• A column comes out as JSON or VARCHAR instead of a struct — the field has mixed shapes across records (an object here, a string there), or the variation sits beyond the inference sample. Pass sample_size = -1 to scan every record, or declare the types yourself:

  read_json('events.json', format = 'array',
            columns = {id: 'BIGINT', user: 'STRUCT(name VARCHAR, plan VARCHAR)',
                       tags: 'VARCHAR[]', ts: 'TIMESTAMPTZ'})
  

  sqlCopy

• IDs lose leading zeros or overflow — values such as "00123" stay strings, but unquoted 20-digit numbers become DOUBLE. Declare them VARCHAR in columns.

• Timestamps stay strings — non-ISO formats are not detected. Pass timestampformat = '%d/%m/%Y %H:%M' or cast with strptime() in the SELECT.

In the browser

Drop the .json file into the SQL Workbench and run the same read_json(..., format = 'array') query to check the inferred schema and spot bad records — the file stays on your machine. The workbench downloads results as CSV; for a direct Parquet output, convert the array to JSONL and use JSONL to Parquet, which infers the schema the same way and writes ZSTD-compressed Parquet locally. Open the result in the Parquet Viewer to confirm nested columns kept their structure.

Going the other direction? See Parquet to JSON.