php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #67394 "Error unserializing raw response" on statistics response having NaN as value
Submitted: 2014-06-06 15:19 UTC Modified: 2017-10-24 06:31 UTC
Votes:4
Avg. Score:4.0 ± 1.0
Reproduced:3 of 3 (100.0%)
Same Version:1 (33.3%)
Same OS:1 (33.3%)
From: ochs at lrz dot de Assigned:
Status: Open Package: solr (PECL)
PHP Version: Irrelevant OS: SLES 11.3
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: ochs at lrz dot de
New email:
PHP Version: OS:

 

 [2014-06-06 15:19 UTC] ochs at lrz dot de
Description:
------------
If using the stats component and solr returns a "stddev" value having "NaN", getResponse() crashes with Error: PHP Fatal error:  Uncaught exception 'SolrException' with message 'Error un-serializing response'.

Solr Response which won't crash getResponse():
{
  "responseHeader": {
    "status": 0,
    "QTime": 1,
    "params": {
      "fl": "date",
      "indent": "true",
      "start": "0",
      "stats": "true",
      "stats.field": "date",
      "q": "ochs",
      "_": "1402067544083",
      "wt": "json",
      "rows": "1"
    }
  },
  "response": {
    "numFound": 99,
    "start": 0,
    "docs": [
      {
        "date": "2014-03-13T14:41:56Z"
      }
    ]
  },
  "stats": {
    "stats_fields": {
      "date": {
        "min": "2013-10-21T12:53:15Z",
        "max": "2014-06-02T09:15:50Z",
        "count": 99,
        "missing": 0,
        "sum": "6339-03-18T07:44:16.999Z",
        "mean": "2014-02-18T09:31:57.747Z",
        "sumOfSquares": 51268826089436770000,
        "stddev": 723625233.6835976,
        "facets": {}
      }
    }
  }
}

Solr Response which crashes getResponse():
{
  "responseHeader": {
    "status": 0,
    "QTime": 5,
    "params": {
      "fl": "date",
      "indent": "true",
      "start": "0",
      "stats": "true",
      "stats.field": "date",
      "q": "bahruz",
      "_": "1402067668264",
      "wt": "json",
      "rows": "1"
    }
  },
  "response": {
    "numFound": 1039,
    "start": 0,
    "docs": [
      {
        "date": "2012-10-10T15:05:18Z"
      }
    ]
  },
  "stats": {
    "stats_fields": {
      "date": {
        "min": "2011-10-20T11:16:34Z",
        "max": "2014-06-04T13:41:45Z",
        "count": 1039,
        "missing": 0,
        "sum": "47085-11-30T17:59:27.999Z",
        "mean": "2013-06-03T17:17:43.684Z",
        "sumOfSquares": -234260129500201700000,
        "stddev": "NaN",
        "facets": {}
      }
    }
  }
}


Test script:
---------------
$solr_client = new SolrClient($options);
$solr_query = new SolrQuery();
$solr_query->setQuery( "bahruz" );
$solr_query->setStart( 0 );
$solr_query->setRows( 10 );					
$solr_query->addParam( "stats", "true" );
$solr_query->addParam( "stats.field", "date" );

$solr_response = $solr_client->query( $solr_query );
$response = $solr_response->getResponse();


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2014-06-07 14:43 UTC] omars@php.net
-Assigned To: +Assigned To: omars
 [2014-06-09 13:21 UTC] omars@php.net
The posted JSON string seems valid.

I'm unable to reproduce the error from Solr. So I enforced the json string to be decoded, but It didn't produce any errors.

Can you please try to decode the Erronemous JSON string using json_decode() and let me know the result.

Please also mention the following:
- PHP Version
- PECL Solr Extension Version
- Solr Server Version
- Whether you are using standard php json extension or jsonc pecl extension ? 
- Is there any other warnings produced ?

Solr Server previously had an issue where the NaN was not enclosed by quotes which resulted in parse errors, not sure if that's relevant. Since your json output contains a quoted NaN.
 [2014-06-10 12:57 UTC] ochs at lrz dot de
Hi, 
thank you for your assistance.
I tried json encoding of the solr return string without producing any errors.

When I use $solr_response->getResponse(); without assigning the return object to any variable the script runs through (of course the following warning is produced: PHP Notice:  SolrResponse::getResponse(): Return value requested without processing output). But when I assign the return value to a variable like $response = $solr_response->getResponse(); this will produce the exception.

Besides, I also experimented a little with the response writers. The exception does not occur using "phpnative" (client side) and "org.apache.solr.response.PHPSerializedResponseWriter" (server side).

In addition, these are the outputs produced by several scenarios I tested:

Output of getRawResponse using $solr_client->setResponseWriter("xml");
0 2 on 0 date true bahruz phpnative 0 2.2 2011-10-20T11:16:34Z 2014-06-04T13:41:45Z 1039 0 47085-11-30T17:59:27.999Z 2013-06-03T17:17:43.684Z -2.342601295002017E20 NaN

Output of getRawResponse using $solr_client->setResponseWriter("phpntive");
a:3:{s:14:"responseHeader";a:3:{s:6:"status";i:0;s:5:"QTime";i:3;s:6:"params";a:8:{s:6:"indent";s:2:"on";s:5:"start";s:1:"0";s:11:"stats.field";s:4:"date";s:5:"stats";s:4:"true";s:1:"q";s:6:"bahruz";s:2:"wt";s:9:"phpnative";s:4:"rows";s:1:"0";s:7:"version";s:3:"2.2";}}s:8:"response";a:3:{s:8:"numFound";i:1039;s:5:"start";i:0;s:4:"docs";a:0:{}}s:5:"stats";a:1:{s:12:"stats_fields";a:1:{s:4:"date";a:9:{s:3:"min";s:20:"2011-10-20T11:16:34Z";s:3:"max";s:20:"2014-06-04T13:41:45Z";s:5:"count";i:1039;s:7:"missing";i:0;s:3:"sum";s:25:"47085-11-30T17:59:27.999Z";s:4:"mean";s:24:"2013-06-03T17:17:43.684Z";s:12:"sumOfSquares";d:-2.342601295002017E20;s:6:"stddev";s:3:"NaN";s:6:"facets";a:0:{}}}}}

Output of json_encode(getResponse()) using $solr_client->setResponseWriter("phpntive");
Using "phpnative" as serialized response writer (org.apache.solr.response.PHPSerializedResponseWriter) -> encoding to json:
{"responseHeader":{"status":0,"QTime":3,"params":{"indent":"on","start":"0","stats.field":"date","stats":"true","q":"bahruz","wt":"phpnative","rows":"0","version":"2.2"}},"response":{"numFound":1039,"start":0,"docs":[]},"stats":{"stats_fields":{"date":{"min":"2011-10-20T11:16:34Z","max":"2014-06-04T13:41:45Z","count":1039,"missing":0,"sum":"47085-11-30T17:59:27.999Z","mean":"2013-06-03T17:17:43.684Z","sumOfSquares":-2.342601295002e+20,"stddev":"NaN","facets":[]}}}}

Output of json_encode() using the standard xml reply from solr I already posted above.
Encoding xml response directly to json with json_encode:
{"lst":[{"@attributes":{"name":"responseHeader"},"int":["0","2"],"lst":{"@attributes":{"name":"params"},"str":["on","0","bahruz","true","date","xml","2.2","0"]}},{"@attributes":{"name":"stats"},"lst":{"@attributes":{"name":"stats_fields"},"lst":{"@attributes":{"name":"date"},"date":["2011-10-20T11:16:34Z","2014-06-04T13:41:45Z","47085-11-30T17:59:27.999Z","2013-06-03T17:17:43.684Z"],"long":["1039","0"],"double":["-2.342601295002017E20","NaN"],"lst":{"@attributes":{"name":"facets"}}}}}],"result":{"@attributes":{"name":"response","numFound":"1039","start":"0"}}}

---

System details:
PHP: PHP 5.3.17 (cli)-> I know this is not the newest build but I am not allowed to upgrade because SLES 11.3 supports PHP 5.3.17 officially. Do you think this is a PHP version problem?
---
PHP-Solr: PECL solr-2.0.0b
---
Solr Server: Solr 7.6.1 on remote machine
---
standard php json extension: 1.2.1
---
Complete error message (there are no more warnings reported): 
[Tue Jun 10 14:06:09 2014] [error] [client 2001:4ca0:0:f000:ec0d:6bac:bb84:e6a0] PHP Warning:  SolrResponse::getResponse(): Error unserializing raw response. in testscript.php on line 50
[Tue Jun 10 14:06:09 2014] [error] [client 2001:4ca0:0:f000:ec0d:6bac:bb84:e6a0] PHP Fatal error:  Uncaught exception 'SolrException' with message 'Error un-serializing response' in testscript.php:50\nStack trace:\n#0 testscript.php(50): SolrResponse->getResponse()\n#1 {main}\n  thrown in testscript.php on line 50
 [2015-09-22 15:37 UTC] nobody at nobody dot de
Hi there,

I have a similar problem. The stats component is activated, which seems to trigger the bug.

I get the following response (stats-part), if there are no items in the resultset:

<lst name="stats">
  <lst name="stats_fields">
    <lst name="currentPrice">
      <null name="min"/>
      <null name="max"/>
      <long name="count">0</long>
      <long name="missing">0</long>
      <double name="sum">0.0</double>
      <double name="sumOfSquares">0.0</double>
      <double name="mean">NaN</double>
      <double name="stddev">0.0</double>
    </lst>
  </lst>
</lst>

The NaN seems to be the evil player.

I am using solr 5.3.0, And pecl-solr 2.1.0.
 [2015-09-24 17:45 UTC] omars@php.net
The issue with xml response writer is totally valid as it is declared as double, however I still cannot reproduce using json response writer.

using the xml response writer I have fixed it to be represented as php NAN

As of phps response writer, there is a bug 
org.apache.solr.response.PHPSerializedResponseWriter as it returns it as a string, while php serialization allows NAN. That should be fixed from Solr Server.

As for json, according to the json spec
"Numeric values that cannot be represented as sequences of digits
   (such as Infinity and NaN) are not permitted."

http://www.ietf.org/rfc/rfc4627.txt

So currently it is represented as "NaN" string, with no exceptions produced, its properly decoded and I think that is acceptable.

So as of the next release 2.2.0, the type of NaN will handled differently with different response writers:
phps: string "NaN"
xml: float NAN
json: string "NaN"
 [2017-10-24 06:31 UTC] kalle@php.net
-Status: Assigned +Status: Open -Assigned To: omars +Assigned To:
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Sun Nov 19 01:31:42 2017 UTC