Python Selen Test wird in urlopen

Question

Meine App stecken verlässt sich auf:

• Python 3
• Django 1.8
• Weasyprint
• Selen

Es läuft einwandfrei auf Entwickler und Produktionsumgebung , aber nicht während des Tests mit Selen.

Mit weasyprint, ich erstelle eine PDF aus HTML, diese Bibliothek verwendet Urlib zum Download von CSS (z. B. http://localhost:8081/static/lib/bootstrap/css/bootstrap.min.css), aber es hängt (keine Fehler, nur stecken) beim Öffnen dieser.

Wenn ich diese URL direkt in meinem Browser während gehängt eingeben, wird das CSS angezeigt.

Befehl verwendet:

./manage.py test tests.test_account.HomeNewVisitorTest 

Relevante Teil des Tests:

from selenium import webdriver 

class HomeNewVisitorTest(StaticLiveServerTestCase): 
    def setUp(self): 
     if TEST_ENV_FIREFOX: 
      self.driver = webdriver.Firefox() 
     else: 
      self.driver = webdriver.PhantomJS() 
     self.driver.set_window_size(1440, 900) 

    def tearDown(self): 
     try: 
      path = 'worksites/' + self.worksite_name.lower() 
      os.rmdir(settings.MEDIA_ROOT + path) 
     except FileNotFoundError: 
      pass 
     super().tearDown() 

    def test(self): 
     d = self.driver 
     d.get(self.get_full_url('home')) 
     d.find_element_by_css_selector('.btn-success[type=submit]').click() 

Aus meiner Sicht:

# Generate PDF for contact directory 
    template = get_template("pdf/annuaire.html") 
    context = {"worksite": worksite} 
    html = template.render(RequestContext(self.request, context)) 
    base_url = self.request.build_absolute_uri("/") 
    pdf = weasyprint.HTML(string=html, base_url=base_url) 
    pdf.write_pdf(directory + '/annuaire.pdf') 

Hier ist das Thread-Dump, wenn stecken:

Fatal Python error: Aborted 

Thread 0x0000000106f92000 (most recent call first): 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374 in readinto 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313 in _read_status 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351 in begin 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171 in getresponse 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185 in do_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210 in http_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441 in _call_chain 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481 in _open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463 in open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 161 in urlopen 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/urls.py", line 276 in default_url_fetcher 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/urls.py", line 311 in fetch 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/contextlib.py", line 59 in __enter__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 297 in _select_source 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/contextlib.py", line 59 in __enter__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 223 in __init__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/css/__init__.py", line 198 in find_stylesheets 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/css/__init__.py", line 448 in get_all_computed_styles 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/document.py", line 312 in _render 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 132 in render 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 164 in write_pdf 
    File "/Users/sebcorbin/Sites/planexo/worksite/views.py", line 111 in done 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 357 in render_done 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 730 in render_done 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 300 in post 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 686 in post 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/views/generic/base.py", line 89 in dispatch 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 237 in dispatch 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/views/generic/base.py", line 71 in view 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/contrib/auth/decorators.py", line 22 in _wrapped_view 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/handlers/base.py", line 132 in get_response 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/handlers/wsgi.py", line 189 in __call__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/testcases.py", line 1099 in __call__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/contrib/staticfiles/handlers.py", line 63 in __call__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/wsgiref/handlers.py", line 137 in run 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/servers/basehttp.py", line 182 in handle 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 673 in __init__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/servers/basehttp.py", line 102 in __init__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 344 in finish_request 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 331 in process_request 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 305 in _handle_request_noblock 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 238 in serve_forever 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/testcases.py", line 1182 in run 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 920 in _bootstrap_inner 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 888 in _bootstrap 

Current thread 0x00007fff7996a300 (most recent call first): 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374 in readinto 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313 in _read_status 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351 in begin 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171 in getresponse 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185 in do_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210 in http_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441 in _call_chain 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481 in _open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463 in open 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 457 in _request 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 389 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 191 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/webelement.py", line 447 in _execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/webelement.py", line 68 in click 
    File "/Users/sebcorbin/Sites/planexo/tests/test_account.py", line 203 in _test_worksite_form 
    File "/Users/sebcorbin/Sites/planexo/tests/test_account.py", line 36 in test 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 577 in run 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 625 in __call__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/testcases.py", line 186 in __call__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/suite.py", line 122 in run 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/suite.py", line 84 in __call__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/runner.py", line 168 in run 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/runner.py", line 178 in run_suite 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/runner.py", line 211 in run_tests 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/commands/test.py", line 90 in handle 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/base.py", line 441 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/commands/test.py", line 74 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/base.py", line 390 in run_from_argv 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/commands/test.py", line 30 in run_from_argv 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/__init__.py", line 330 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/__init__.py", line 338 in execute_from_command_line 
    File "./manage.py", line 10 in <module> 

Answer 1

Dank an euch alle, das Problem bei der genauen Erkennung, die ich für Inspiration nahm diesen Artikel http://nedbatchelder.com/blog/201103/quick_and_dirty_multithreaded_django_dev_server.html und modifizierte meine manage.py Datei:

#!/usr/bin/env python 
from http import server 
import os 
from socketserver import ThreadingMixIn 
import sys 


def monkey_patch_test_server(): 
    # This monkey-patches HTTPServer to create a base HTTPServer class that 
    # supports multithreading 
    originalhttpserver = server.HTTPServer 

    class ThreadedHTTPServer(ThreadingMixIn, originalhttpserver): 
     def __init__(self, server_address, RequestHandlerClass=None): 
      originalhttpserver.__init__(self, server_address, 
             RequestHandlerClass) 

    server.HTTPServer = ThreadedHTTPServer 


if __name__ == "__main__": 
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "planexo.settings") 

    from django.core.management import execute_from_command_line 

    if sys.argv[1] == 'test': 
     monkey_patch_test_server() 

    execute_from_command_line(sys.argv) 

Jetzt funktioniert es!

Answer 2

Verwenden Sie Python faulthandler Modul, um die Traceback aller hängen Threads zu drucken. Dies wird Ihnen die eigentlichen Funktionen zeigen, wo das Aufhängen stattfindet.

Bei Bedarf bearbeiten Sie Ihre Frage und fügen Sie die relevanten Informationen hinzu.

Ansonsten denke ich, dass Sie den Django Test-/Entwicklungsserver so verwenden, dass er nur eine Anfrage gleichzeitig verarbeiten kann. Ihre anfängliche HTTP-Anfrage an den Dev-Server löst eine Anfrage aus, um Daten von sich selbst anzufordern, die der Django-Dev-Server nicht ausführen kann. Aber das ist nur eine Vermutung.

Auch relevante Quellcode fehlt in der Frage.

Answer 3

Verwenden Sie das Modul faulthandler. Vielleicht kann es auch sein, dass Django nur 1 Anfrage annehmen kann.

Answer 4

Ich werde meinen Fall teilen:

Ich hatte dieses Problem mit meinem Django Selen-Tests (django == 1.7.12 und Selen == 2.53.1) mit ChromeDriver 2.21.371459 und Google Chrome 48.0.2564.116 .

Ich konnte das Problem isolieren. In meinem Fall geschah dies nur für Seiten, die auf eine statische Datei verweisen (ein Bild in einem HTML-Code , zum Beispiel http://cdn.local.myproject.net/static/myimage.png) in meiner benutzerdefinierten lokalen cdn-Domäne. Das Problem trat nicht auf, wenn ich einen relativen Pfad "/ static/myimage" verwendete.png“oder localhost "http://127.0.0.1/static/myimage.png" so habe ich es ein DNS-Problem war

ich war in der Lage, das Problem zu umgehen, indem sie mit der --dns-prefetch-disable Option aus Chrom

Beispiel in Python:..

from selenium.webdriver import Chrome 
from selenium.webdriver.chrome.options import Options 

chrome_options = Options() 
chrome_options.add_argument('--dns-prefetch-disable') 
driver = Chrome(chrome_options=options) 

Ich weiß nicht, ob das der allgemeine Fall ist, aber hoffentlich kann es einigen von Ihnen helfen.