Array serialization truncation too aggressive

[benvinegar]

All arrays are currently being truncated to "Array of length N":

sentry-php/lib/Raven/ReprSerializer.php

Line 35 in c8bc22e

} elseif (is_array($value)) {

... and yet dictionaries are fully expanded. We are getting feedback from users with error data containing small arrays (e.g. 1-5 items) that are frustrated that they cannot see inside.

[lupoair]

Yes, frustrating!

[dcramer]

I'm closing this out as we expand arrays in PHP.

[idoberko2]

This issue still exists. Will it be fixed?

[dcramer]

Just ton confirm, where are these arrays not getting expanded? (which section of the event)

[idoberko2]

Additional Data. For example, we receive JSON webhooks from 3rd parties and send events to sentry in case of failure. Attached an example of how one of these JSONs look in Sentry:

[Jean85]

We should anyway limit the depth or let it be configurable, to avoid sending data too "deep".

[dcramer]

I'm trying to recall why we went this direction in the first place. I think there were certain things where we didn't want the full data repr, but honestly dont recall where.

[rentalhost]

In my case, this additional data are required to make possible fix my issue, once that the problem is inside of this truncated array.

[pontus-mp]

According to support this feature is intended to help keep error messages within the 100kb payload limit.

Maybe it would be possible to only flatten when the limit is hit, or make it opt-out? Those who are migrating from another solution (like us) are probably already having their error messages somewhat in check.

[Jean85]

In the 1.8.1 release that just got out, we added setters for the serializers. You can replace them with the logic you prefer.

[gsouf]

Hi @Jean85 can you give details about this method?

In my case the data truncated from $_SESSION are required to reproduce the error. To workarround the depth limit, I currently send a json encoded copy of $_SESSION (with pretty print) in extra param.

[Jean85]

I'm referencing this piece of code:

sentry-php/lib/Raven/Client.php

Lines 1457 to 1471 in e1001f8

	/**
	* @param Raven_Serializer $serializer
	*/
	public function setSerializer(Raven_Serializer $serializer)
	{
	$this->serializer = $serializer;
	}
	/**
	* @param Raven_ReprSerializer $reprSerializer
	*/
	public function setReprSerializer(Raven_ReprSerializer $reprSerializer)
	{
	$this->reprSerializer = $reprSerializer;
	}

Those 2 serializers are the classes responsible for the compression of those arrays. You can extend them and change the behavior for the desired stuff. But using the extra field like that is a feasible solution too!

[gsouf]

Thanks @Jean85
Unfortunately setting the serializer was not useful in my case because the depth is set in Client::sanitize and wanted to set a greater depth for user data only.

I end up by ovewritting sanitize method to do the job. Here is an example in case it can help anyone:

class MyRavenClient extends \Raven_Client
{

    /**
     * @param array $data
     */
    public function sanitize(&$data)
    {
        // Serializes the session items as a json string
        foreach ($data['user']['data'] as $k => $v) {
            if (is_array($v)) {
                foreach ($v as $kk => $vv) {
                    $data['user']['data'][$k][$kk] = json_encode($vv, JSON_PRETTY_PRINT, 3);
                }
            }
        }


        // use parent sanitizer
        parent::sanitize($data);
    }

}

Be aware that serializer will limit the size of a string, and in this case it might be useful to provide an other serializer that has a greater limit on string size.

[Wirone]

Also having this:

This relates to issues created via overriden Monolog's RavenHandler in Yii2 app (don't ask....).

I'll probably normalize trace to flat array, but still, this is frustrating.

[philraj]

Any movement on this? It would be great if serialization depth was configurable for each of the types of data this client attaches to Sentry events (user, extra, tags...).

[Jean85]

Did you check out the new 2.0 release? The serializer has that option and it's completely replaceable if you want.

Also, for 1.x there's #632 which just needs to be released.

[philraj]

@Jean85 that's great news, thank you. Will definitely be trying that out.

[stayallive]

This should be resolved in both 1.x and 2.x (but please use 2.x where possible).

Or well, the "culprit" is now configurable so if you run into this often it's possible to modify the settings for this behaviour.

1.x: #632
2.x: #378 (comment)