Python requests.exceptions.ChunkedEncodingError

Question

Ich schreibe ein Python-Skript, das die folgenden Schritte ausführt.

• Abfrage eine MongoDB Datenbank
• Parse und Gesamtergebnisse
• Hochladen von Daten in eine ServiceNow Tabelle über eine REST-API

Dies funktioniert die meiste Zeit, aber gelegentlich sehe ich diesen Fehler:

requests.exceptions.ChunkedEncodingError: ("Connection broken: error(104, 'Connection reset by peer')", error(104, 'Connection reset by peer')) 

Dieser Fehler stoppt das Skript und verhindert, dass der gesamte Datensatz erfasst wird.

Was kann ich tun, um dieses Problem zu beheben?

Python 2.7.5

-Code

#!/usr/bin/env python 

from config import * 

import os, sys 

mypath = os.path.dirname(os.path.realpath(__file__)) 
sys.path.append(os.path.join(mypath, "api-python-client")) 

from apiclient.mongo import * 

from pymongo import MongoClient 

import json 

import requests 

from bson.json_util import dumps 

client = MongoClient(mongo_uri) 

#Create ServiceNow URL 
svcnow_url = create_svcnow_url('u_imp_cmps') 

#BITSDB Nmap Collection 
db = client[mongo_db] 

#Aggregate - RDBMS equivalent to Alias select x as y 
#Rename fields to match ServiceNow field names 
computers = db['computer'].aggregate([ 
     {"$unwind": "$hostnames"}, 
     {"$project" : { 
       "_id":0, 
       "u_hostname": "$hostnames.name", 
       "u_ipv4": "$addresses.ipv4", 
       "u_status": "$status.state", 
       "u_updated_timestamp": "$last_seen" 
     }} 

]) 

j = dumps({"records":computers}) 
#print(j) 


#Set proper headers 
headers = {"Content-Type":"application/json","Accept":"application/json"} 

#Build HTTP Request 
response = requests.post(url=svcnow_url, auth=(svcnow_user, svcnow_pwd), headers=headers ,data=j) 

#Check for HTTP codes other than 200 
if response.status_code != 200: 
     print('Status:', response.status_code, 'Headers:', response.headers, 'Response Text', response.text, 'Error Response:',response.json()) 
     exit() 

#Decode the JSON response into a dictionary and use the data 
print('Status:',response.status_code,'Headers:',response.headers,'Response:',response.json()) 

Fehler

Traceback (most recent call last): 
    File "/usr/src/computer_pingable_import.py", line 50, in <module> 
    response = requests.post(url=svcnow_url, auth=(svcnow_user, svcnow_pwd), headers=headers ,data=j) 
    File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 107, in post 
    return request('post', url, data=data, json=json, **kwargs) 
    File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 53, in request 
    return session.request(method=method, url=url, **kwargs) 
    File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 468, in request 
    resp = self.send(prep, **send_kwargs) 
    File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 608, in send 
    r.content 
    File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 737, in content 
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes() 
    File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 663, in generate 
    raise ChunkedEncodingError(e) 
requests.exceptions.ChunkedEncodingError: ("Connection broken: error(104, 'Connection reset by peer')", error(104, 'Connection reset by peer')) 

Ein weiterer Versuch, und eine etwas andere Fehlermeldung:

Traceback (most recent call last): 
    File "/usr/src/computer_pingable_import.py", line 50, in <module> 
    response = requests.post(url=svcnow_url, auth=(svcnow_user, svcnow_pwd), headers=headers ,data=j) 
    File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 107, in post 
    return request('post', url, data=data, json=json, **kwargs) 
    File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 53, in request 
    return session.request(method=method, url=url, **kwargs) 
    File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 468, in request 
    resp = self.send(prep, **send_kwargs) 
    File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 576, in send 
    r = adapter.send(request, **kwargs) 
    File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 426, in send 
    raise ConnectionError(err, request=request) 
requests.exceptions.ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer')) 

Answer 1

ServiceNow prevents REST transactions fr om läuft länger als 60 Sekunden.

Ich bin mir nicht sicher, wie groß Ihr Dataset ist, aber Sie möchten die Daten in kleinere Teile zerlegen, um sicherzustellen, dass die Transaktion immer ausgeführt wird.