ValueError: Could Not Determine the Shape of Object

ValueError: could not determine the shape of object type and its cousin setting an array element with a sequence both mean the same thing: NumPy tried to build a rectangular array from data that is not rectangular. Usually one row has a different length from the others, or the data contains a mix of types NumPy cannot reconcile.

Reproducing it#

import numpy as np

rows = [[1, 2, 3], [4, 5], [6, 7, 8]]
np.array(rows)

ValueError: setting an array element with a sequence. The requested array has an
inhomogeneous shape after 1 dimensions.

A NumPy array is a single block of memory with a fixed shape, which is exactly what makes it fast. Rows of different lengths cannot be laid out that way, so NumPy refuses rather than guessing.

Older NumPy versions created an object array with a warning instead. Version 1.24 made it an error, which is why code that worked last year may not now.

Step 1: find the offending row#

from collections import Counter

lengths = [len(row) for row in rows]
print(Counter(lengths))

expected = Counter(lengths).most_common(1)[0][0]

for i, row in enumerate(rows):
    if len(row) != expected:
        print(f"row {i} has {len(row)} items: {row}")

On real data this usually finds one or two rows out of thousands — a truncated record, a header that slipped through, or a line with a missing field.

Check the types too, since a mix can cause the same error:

types = {type(x) for row in rows for x in row}
print(types)

Fix 1: pad to a common length#

import numpy as np


def pad_rows(rows, fill=np.nan):
    width = max(len(row) for row in rows)
    return np.array([list(row) + [fill] * (width - len(row)) for row in rows],
                    dtype=float)


arr = pad_rows([[1, 2, 3], [4, 5], [6, 7, 8]])
print(arr)

[[ 1.  2.  3.]
 [ 4.  5. nan]
 [ 6.  7.  8.]]

Padding with np.nan requires a float array, which is usually fine. NumPy’s nan-aware functions then skip the padding automatically:

print(np.nanmean(arr, axis=1))
print(np.nansum(arr))

Fix 2: truncate to the shortest#

width = min(len(row) for row in rows)
arr = np.array([row[:width] for row in rows])

Right when the extra elements are genuinely surplus — time series of different lengths where you only want the overlapping period, for instance. Wrong when you would be silently discarding data, so check how much you are dropping.

Fix 3: keep them separate#

arrays = [np.array(row) for row in rows]

means = [a.mean() for a in arrays]
total = sum(a.sum() for a in arrays)

If the rows are genuinely independent measurements of different lengths, forcing them into one rectangle is the wrong model. A list of arrays keeps each one fast and loses nothing.

Fix 4: dtype=object#

arr = np.empty(len(rows), dtype=object)
arr[:] = rows

print(arr.shape)     # (3,)
print(arr[1])        # [4, 5]

This works, and it is worth being clear about what you lose. An object array stores Python object pointers, not numbers, so vectorised arithmetic falls back to a Python loop. You get NumPy’s syntax with a list’s performance.

The mixed-types version#

np.array([1, 2, "three"])
# array(['1', '2', 'three'], dtype='<U21')  - everything became a string

This one does not raise. NumPy finds a type everything fits into, and strings win, so your numbers silently become text. Then arithmetic fails somewhere else entirely:

numpy.core._exceptions.UFuncTypeError: ufunc 'add' did not contain a loop with
signature matching types

Be explicit about the type you expect, so a bad value fails immediately:

arr = np.array(values, dtype=float)     # raises on "three"

Or convert leniently and count the failures:

import pandas as pd

numbers = pd.to_numeric(pd.Series(values), errors="coerce")
print(numbers.isna().sum(), "values could not be converted")
arr = numbers.to_numpy(dtype=float)

The pandas version#

import pandas as pd

pd.DataFrame({"a": [1, 2, 3], "b": [4, 5]})

ValueError: All arrays must be of the same length

Same cause, clearer message. Build from a list of dictionaries instead, and pandas fills the gaps:

df = pd.DataFrame([
    {"a": 1, "b": 4},
    {"a": 2, "b": 5},
    {"a": 3},
])
print(df)      # b is NaN in the third row

The machine learning version#

import torch

torch.tensor([[1, 2, 3], [4, 5]])
# ValueError: expected sequence of length 3 at dim 1

Tensors have the same rectangular requirement. Sequences of different lengths — sentences, time series — are the usual cause, and padding is the standard answer:

from torch.nn.utils.rnn import pad_sequence

sequences = [torch.tensor(row) for row in rows]
padded = pad_sequence(sequences, batch_first=True, padding_value=0)

lengths = torch.tensor([len(r) for r in rows])

Keep the lengths alongside the padded tensor. Without them the model cannot tell padding from real zeros, and it will learn from the padding.

Preventing it#

def to_array(rows, expected_width=None, dtype=float):
    """Build a NumPy array, failing with a useful message."""
    lengths = {len(row) for row in rows}

    if len(lengths) > 1:
        counts = Counter(len(row) for row in rows)
        raise ValueError(f"Rows have different lengths: {dict(counts)}")

    width = lengths.pop()
    if expected_width is not None and width != expected_width:
        raise ValueError(f"Expected width {expected_width}, got {width}")

    return np.array(rows, dtype=dtype)

Validating at the point where the data is built gives you a message naming the problem, rather than a NumPy error several functions later.

Where to go next#

• Exploring np.stack() — which requires identical shapes.
• Python and data science — cleaning data before it reaches NumPy.
• Solving dict TypeError — reading type errors generally.