Verbinden mit einer AWS Elasticsearch-Instanz mit Python

Question

Ich habe eine Elasticsearch-Instanz, die auf AWS gehostet wird. Ich kann von meinem Terminal aus mit Curl verbinden. Ich versuche jetzt, den Python Elasticsearch Wrapper zu verwenden. Ich habe:

from elasticsearch import Elasticsearch 

client = Elasticsearch(host='https://ec2-xx-xx-xxx-xxx.us-west-2.compute.amazonaws.com', port=9200) 

und die Abfrage ist:

data = client.search(index="mynewindex", body={"query": {"match": {"email": "gmail"}}}) 
    for hit in data: 
     print(hit.email) 
    print data 

Die vollständige Rückverfolgung von Heroku ist:

2016-07-22T14:06:06.031347+00:00 heroku[router]: at=info method=GET path="/" host=elastictest.herokuapp.com request_id=9a96d447-fe02-4670-bafe-efba842927f3 fwd="88.106.66.168" dyno=web.1 connect=1ms service=393ms status=500 bytes=456 
2016-07-22T14:09:18.035805+00:00 heroku[slug-compiler]: Slug compilation started 
2016-07-22T14:09:18.035810+00:00 heroku[slug-compiler]: Slug compilation finished 
2016-07-22T14:09:18.147278+00:00 heroku[web.1]: Restarting 
2016-07-22T14:09:18.147920+00:00 heroku[web.1]: State changed from up to starting 
2016-07-22T14:09:20.838784+00:00 heroku[web.1]: Starting process with command `gunicorn application:application --log-file=-` 
2016-07-22T14:09:20.834521+00:00 heroku[web.1]: Stopping all processes with SIGTERM 
2016-07-22T14:09:17.850918+00:00 heroku[api]: Deploy b7187d3 by hector@fastmail.se 
2016-07-22T14:09:17.850993+00:00 heroku[api]: Release v21 created by hector@fastmail.se 
2016-07-22T14:09:21.372589+00:00 app[web.1]: [2016-07-22 14:09:21 +0000] [3] [INFO] Handling signal: term 
2016-07-22T14:09:21.383946+00:00 app[web.1]: [2016-07-22 14:09:21 +0000] [3] [INFO] Shutting down: Master 
2016-07-22T14:09:21.367656+00:00 app[web.1]: [2016-07-22 14:09:21 +0000] [9] [INFO] Worker exiting (pid: 9) 
2016-07-22T14:09:21.366309+00:00 app[web.1]: [2016-07-22 14:09:21 +0000] [10] [INFO] Worker exiting (pid: 10) 
2016-07-22T14:09:22.286766+00:00 heroku[web.1]: Process exited with status 0 
2016-07-22T14:09:23.344822+00:00 app[web.1]: [2016-07-22 14:09:23 +0000] [3] [INFO] Starting gunicorn 19.6.0 
2016-07-22T14:09:23.345481+00:00 app[web.1]: [2016-07-22 14:09:23 +0000] [3] [INFO] Using worker: sync 
2016-07-22T14:09:23.351173+00:00 app[web.1]: [2016-07-22 14:09:23 +0000] [9] [INFO] Booting worker with pid: 9 
2016-07-22T14:09:23.370580+00:00 app[web.1]: [2016-07-22 14:09:23 +0000] [10] [INFO] Booting worker with pid: 10 
2016-07-22T14:09:23.345376+00:00 app[web.1]: [2016-07-22 14:09:23 +0000] [3] [INFO] Listening at: http://0.0.0.0:59867 (3) 
2016-07-22T14:09:24.536725+00:00 heroku[web.1]: State changed from starting to up 
2016-07-22T14:09:39.043240+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception 
2016-07-22T14:09:39.043239+00:00 app[web.1]:  rv = self.handle_user_exception(e) 
2016-07-22T14:09:39.043241+00:00 app[web.1]:  reraise(exc_type, exc_value, tb) 
2016-07-22T14:09:39.043233+00:00 app[web.1]: Traceback (most recent call last): 
2016-07-22T14:09:39.043238+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request 
2016-07-22T14:09:39.043236+00:00 app[web.1]:  response = self.full_dispatch_request() 
2016-07-22T14:09:39.043235+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app 
2016-07-22T14:09:39.043214+00:00 app[web.1]: [2016-07-22 14:09:39,041] ERROR in app: Exception on/[GET] 
2016-07-22T14:09:39.043241+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request 
2016-07-22T14:09:39.043242+00:00 app[web.1]:  rv = self.dispatch_request() 
2016-07-22T14:09:39.043242+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request 
2016-07-22T14:09:39.043243+00:00 app[web.1]:  return self.view_functions[rule.endpoint](**req.view_args) 
2016-07-22T14:09:39.043243+00:00 app[web.1]: File "/app/application.py", line 23, in index 
2016-07-22T14:09:39.043246+00:00 app[web.1]:  return func(*args, params=params, **kwargs) 
2016-07-22T14:09:39.043245+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped 
2016-07-22T14:09:39.043246+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 548, in search 
2016-07-22T14:09:39.043247+00:00 app[web.1]:  doc_type, '_search'), params=params, body=body) 
2016-07-22T14:09:39.043250+00:00 app[web.1]:  status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout) 
2016-07-22T14:09:39.043250+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 105, in perform_request 
2016-07-22T14:09:39.043244+00:00 app[web.1]:  data = client.search(index="mynewindex", body={"query": {"match": {"email": "gmail"}}}) 
2016-07-22T14:09:39.043251+00:00 app[web.1]:  raise ConnectionError('N/A', str(e), e) 
2016-07-22T14:09:39.043249+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/transport.py", line 329, in perform_request 
2016-07-22T14:09:39.043253+00:00 app[web.1]: ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f185a94d8d0>: Failed to establish a new connection: [Errno -2] Name or service not known) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f185a94d8d0>: Failed to establish a new connection: [Errno -2] Name or service not known) 
2016-07-22T14:09:42.692817+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request 
2016-07-22T14:09:42.692816+00:00 app[web.1]:  response = self.full_dispatch_request() 
2016-07-22T14:09:42.692795+00:00 app[web.1]: [2016-07-22 14:09:42,691] ERROR in app: Exception on/[GET] 
2016-07-22T14:09:42.692820+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request 
2016-07-22T14:09:42.692819+00:00 app[web.1]:  reraise(exc_type, exc_value, tb) 
2016-07-22T14:09:42.692819+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception 
2016-07-22T14:09:42.692827+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/transport.py", line 329, in perform_request 
2016-07-22T14:09:42.692828+00:00 app[web.1]:  status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout) 
2016-07-22T14:09:42.692828+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 105, in perform_request 
2016-07-22T14:09:42.692829+00:00 app[web.1]:  raise ConnectionError('N/A', str(e), e) 
2016-07-22T14:09:42.692831+00:00 app[web.1]: ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f185a946d10>: Failed to establish a new connection: [Errno -2] Name or service not known) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f185a946d10>: Failed to establish a new connection: [Errno -2] Name or service not known) 
2016-07-22T14:09:42.692821+00:00 app[web.1]:  rv = self.dispatch_request() 
2016-07-22T14:09:42.692821+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request 
2016-07-22T14:09:42.692822+00:00 app[web.1]:  return self.view_functions[rule.endpoint](**req.view_args) 
2016-07-22T14:09:42.692823+00:00 app[web.1]: File "/app/application.py", line 23, in index 
2016-07-22T14:09:42.692823+00:00 app[web.1]:  data = client.search(index="mynewindex", body={"query": {"match": {"email": "gmail"}}}) 
2016-07-22T14:09:42.692824+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped 
2016-07-22T14:09:42.692814+00:00 app[web.1]: Traceback (most recent call last): 
2016-07-22T14:09:42.692818+00:00 app[web.1]:  rv = self.handle_user_exception(e) 
2016-07-22T14:09:42.692815+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app 
2016-07-22T14:09:42.692825+00:00 app[web.1]:  return func(*args, params=params, **kwargs) 
2016-07-22T14:09:42.692826+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 548, in search 
2016-07-22T14:09:42.692826+00:00 app[web.1]:  doc_type, '_search'), params=params, body=body) 
2016-07-22T14:09:42.685540+00:00 heroku[router]: at=info method=GET path="/" host=elastictest.herokuapp.com request_id=87ae9ec2-edb6-4e58-b9d6-89709b883091 fwd="88.106.66.168" dyno=web.1 connect=1ms service=11ms status=500 bytes=456 

Ich nehme an, der Fehler mit der "Verbindungszeichenfolge" ist da der prinzipielle Fehler scheint zu sein ConnectionError

So zwei Fragen:

1) Wie kann ich richtig verbinden? Eingehende Sicherheitsregeln sind derzeit so konfiguriert, dass sie den gesamten eingehenden Datenverkehr akzeptieren

2) Gibt es einen Fehler im Abfragecode?

Vielen Dank wie immer.

Answer 1

Dies ist der richtige Weg, um Elasticsearch Server mit Python verbinden:

es = Elasticsearch(['IP:PORT',]) 

Elasticsearch Konstruktor nicht die host noch die port Parameter hat. Der erste Parameter soll eine Liste sein, wobei jedes Element in der Liste kann entweder eine Zeichenfolge, die den Wirt sein:

'schema://ip:port' 

Oder ein Wörterbuch mit erweiterten Parametern in Bezug auf diesen Host

{'host': 'ip/hostname', 'port': 443, 'url_prefix': 'es', 'use_ssl': True} 

---

In Ihr Fall würden Sie wahrscheinlich verwenden mögen:

client = Elasticsearch(['https://ec2-xx-xx-xxx-xxx.us-west-2.compute.amazonaws.com:9200']) 

The port is redundant since you are using the deafult one, so you can use remove it
client = Elasticsearch(['https://ec2-xx-xx-xxx-xxx.us-west-2.compute.amazonaws.com'])