jellyfin-kodi/resources/lib/libraries/requests/adapters.py

454 lines
17 KiB
Python
Raw Normal View History

Version 4.0.0 (#182) * Adjust refresh behavior * Fix favorites * Add option to mask info * Fix keymap delete * Fix empty show * Version bump 3.1.14 * Reset rescan flag * Fix subtitles encoding * Fix path verification * Fix update library Plug in remove library percentage * Fix unauthorized behavior Reprompt user with login * Fix series pooling * Version bump 3.1.15 * Fix for additional users Return all users, not just public users * Fix http potential errors Prevent from going further if {server} or {userid} is requested but not filled to avoid 401 errors * Fix extra fanart * Fix patch make a case insensitive search * Version bump 3.1.16 Additional logging, fix kodi source. * Fix library tags on update * Version bump 3.1.17 * Fix season artwork * Fix season artwork * Fix logging * Fix blank files sources * Add backup option * Fix userdata song * Transfer data.txt to data.json Use default port for webserver caching * Fix mixed content shortcut * Fix path encoding for patch Hopefully this works... * Fix source nonetype error Just incase, wrap in a try/except because it's not important. * Base fast sync on server time Try to fix music video refresh to prevent cursor from moving up. * Prep subfolders for dynamic Support homevideos for now * Fix empty artist, missing Title * Version bump 3.1.18a * Version bump for objects 171076013 * Notify user of large updates Give option to back out if the user wants to manually update the libraries * Fix sources.xml verification * Prevent error in monitor Put in place try/except in case data is None * Remember sync position for manual triggers Allow to resume sync on restart for manual user triggers (update, repair). Automatically refresh boxsets if movie library is selected. use waitForAbort and emby_should_stop prop to terminate threads * Update string for sync later * Add subfolders for dynamic movies * Small fixes * Version bump 3.1.19 * Fix fast sync try/except, default back to previous behavior. * Fix artwork * Change settings name To ensure it takes default value instead of previous value set in 3.0.34 * Fix transcode flac and live tv * Fix episodes for series pooling * Add live tv support * Version bump 3.1.20 * Revert "Small fixes" This reverts commit 9ec1fa35853352bb6c8a56da94e8c8bcc3843c07. * Version bump 3.1.21 * Fix playback starting server connection instance * Fix show update * Fix boxsets * Fix lastplayed * Patch to support pre 3.6 libraries * Fix slowness * Plug in settings for threading * Plug in settings for threading * Adjust sleep behavior * Version bump 3.1.22 * Fix server detection in monitor * Version bump 3.1.23 * Fix potential error with checksum * Fix missing new artists * Fix library sync Adjust lock, re-add screensaver deactivated during sync, prep compare sync, stop library updates from being processed before startup sync is completed * Version bump 3.1.25 * Fix local trailers * Adjust lock modification * Check db version * Prevent error from creating nodes The addon automatically creates nodes at startup with prefilled information. Prevent errors in the event something goes wrong. It will fix itself down the line, after user has logged in. * Version bump 3.1.26 * Revert "Version bump 3.1.26" This reverts commit c583a69a4b9bb8af160cfb564485cc59da20fca7. * Fix screensaver toggle * Fix source selection for direct stream * Version bump 3.1.26 * Add progress for updates * Revise progress bar Fix typos and subsetting * content notification * Remove content with update library Now remove irrelevant content as well * Fix slowness * Version bump 3.1.27 * Stop trying to get items if server offline * Fix content type for dynamic music * Fix resume sync Now save progress, unless exited due to path validation * Fix artwork for shortcuts on profile switch * Add force transcode settings * Fix audiobooks back to video type Add shortcuts. Audiobook can't be music type otherwise it break resume behavior and it won't play the right item. Has to be video type. * Update general info To finish, download and installation * Update README.md * Move welcome message to service * Prevent patch loop Try once, then let it go, to avoid locking user in a restart loop * Review library threads * Prep for audiobook transcode Still need to implement universal for audio transcode * Version bump 3.1.28 * Fix emby database locked * Fix regression to welcome message * Version bump 3.1.29 * Adjust playback Allow direct play for http streams * Ensure all threads are terminated correctly * Fix empty results due to error 500 * Fix boxset refresh * Fix resume sync behavior Allow to complete the startup sync in the event user backs out of resume sync * Version bump 3.1.30 * Update patch Move patch from cache to addon_data. No longer need to restart Kodi to apply the first patch. * Fix inital sync leading to fast sync * Fix user settings Due to api change in 3.6.0.55 * krypton update * Adjust for resume settings With .55 the resume setting is set per library. Instead query server to see if the item is played to offer delete * Restart service upon check for updates To reload the new objects module. * Fix update library Only do the compare when user selects update library, also add a restart service option in the add-on settings > advanced * Version bump 3.1.31 * Update dependencies * Update FR translation * Update DE translation * Add translation * Support up next * Small service adjustment * Krypton update to support upnext * Add a verification onwake Somehow, Kodi can trigger OnWake without first trigger OnSleep. * Fix loading if special char in path * Add logging and small fixes Prepare userdata by date modified * Version bump 3.1.32 * Change default behavior of startup dialog In case it is forced closed by Kodi, allow the sync to proceed * Ensure deliveryurl is an actual url * Update README.md * Fix nextup * Fix dynamic widgets * Detect coreelect, etc * Fix progress report Silent RefreshProgress in websocket * Follow emby settings for subtitles * Version bump 3.1.33 * Add Italian translation * Fix playback for server 3.6.0.61 * Version bump 3.1.34a * Add silent catch for errors * Adjust playback progress monitor Only track progress report if the item is an emby item * Fix subtitles not following server settings * Add remove libraries, fix mixed libraries * Fix live tv For now, use transcode since direct play returns a 127.0.0.1 unusable address as the path. * Allow live tv to direct stream * Fix LiveTV * Add setting to sync during playback * Fix updates * Fix encoding error * Add optional rotten tomatoes option * Version bump 3.1.35 * Fix emby connect auth string Was preventing proper device detection when using emby connect, play to, etc. * Add setup RT * Fix audio/sub change Only for addon playback * Add developer mode * Update patch Check for updates + dev = forced grab from github * Fix RT string * Fix patch Allow dev mode to redownload zip * Fix patch ugh sleep!! * Verify patch connection * Version bump 3.1.36 * Fix libraries being wiped Catch errors to prevent false positive * Add dateutil library * Prep convert to local time * Fix string * Prep for multi db version support * Fix service restart * Add shortcut restart addon Add notification * Add database discovery * Ensure previous playback terminated * Update translation New: Polish, Dutch Updated: German, French, Italian * Version bump 3.1.37 * Quick fix for new library dateutil * Catch error for dateutil In the event the server has some weird date that can't be converted * Version bump 3.1.38 * Fix dateutil import * Fix db discovery Ignore emby.db * Version bump 3.1.39 * Add a delay if setup not completed Avoid crash from everything loading at once. * Fix database discovery Add table verification + date modified verification * Container optional playutils * Version bump 3.1.40 * Adjust database discovery Compare loaded vs discovered to avoid loading old databases by accident. * Version bump 3.1.41 * Fix discovery toggle * Version bump 3.1.42 * Add webservice for playback prep * Fix service restart * Version bump 3.1.43 * Update default sync indicator Based on overall feedback * Fix check update * Fix if server is selected but unavailable * Support songs without albums * Fix encode and params * Increase retry timeout * Fix update generating duplicates * Add manage libraries Too many entries * Fix database discovery * Fixed transcode via context menu * Fix context transcode * Quiet webservice * Update Krypton objects * Fix database discovery prompt * fixed video listitem issues for krypton * load all item details for playlists * Fix playlist * Version bump 3.1.44 * Fix force hi10p transcoding behavior Fixes the "Force Hi10p transcoding" option to only apply to h264 video codecs * Clear playlist on player.onstop * Don't clear playlist if busy spinner is active * Fix case sensitive issue at calling the log function * fix db stuff (#164) * Reload objects upon initial setup * Fix database discovery ignore db-journal * Update translation German, Italian * Use LastConnectionMode for server test * Fix compare sync * Version bump 3.1.45 * Ensure widgets get updated Container.Refresh alone doesn't seem to work * Update database discovery * Re-add texture to database discovery * Add option to enable/disable service * Remove unused strings * Fix object reload upon restart service * Update Krypton objects * Update translation Dutch, Polish * Version bump 3.1.46 * Adjust client api * Adjust subtitles behavior * Fix string typo * Only run one full sync instance Prevent user from launching multiple syncs and freezing the add-on. * added "playlists" to wnodes * Disable Audiobooks Server doesn't have a set structure yet. This feature is broken atm. * Version bump 4.0.0 * License GPL v3 * Update readme
2019-01-24 13:04:48 +00:00
# -*- coding: utf-8 -*-
"""
requests.adapters
~~~~~~~~~~~~~~~~~
This module contains the transport adapters that Requests uses to define
and maintain connections.
"""
import os.path
import socket
from .models import Response
from .packages.urllib3.poolmanager import PoolManager, proxy_from_url
from .packages.urllib3.response import HTTPResponse
from .packages.urllib3.util import Timeout as TimeoutSauce
from .packages.urllib3.util.retry import Retry
from .compat import urlparse, basestring
from .utils import (DEFAULT_CA_BUNDLE_PATH, get_encoding_from_headers,
prepend_scheme_if_needed, get_auth_from_url, urldefragauth,
select_proxy)
from .structures import CaseInsensitiveDict
from .packages.urllib3.exceptions import ClosedPoolError
from .packages.urllib3.exceptions import ConnectTimeoutError
from .packages.urllib3.exceptions import HTTPError as _HTTPError
from .packages.urllib3.exceptions import MaxRetryError
from .packages.urllib3.exceptions import NewConnectionError
from .packages.urllib3.exceptions import ProxyError as _ProxyError
from .packages.urllib3.exceptions import ProtocolError
from .packages.urllib3.exceptions import ReadTimeoutError
from .packages.urllib3.exceptions import SSLError as _SSLError
from .packages.urllib3.exceptions import ResponseError
from .cookies import extract_cookies_to_jar
from .exceptions import (ConnectionError, ConnectTimeout, ReadTimeout, SSLError,
ProxyError, RetryError)
from .auth import _basic_auth_str
DEFAULT_POOLBLOCK = False
DEFAULT_POOLSIZE = 10
DEFAULT_RETRIES = 0
DEFAULT_POOL_TIMEOUT = None
class BaseAdapter(object):
"""The Base Transport Adapter"""
def __init__(self):
super(BaseAdapter, self).__init__()
def send(self):
raise NotImplementedError
def close(self):
raise NotImplementedError
class HTTPAdapter(BaseAdapter):
"""The built-in HTTP Adapter for urllib3.
Provides a general-case interface for Requests sessions to contact HTTP and
HTTPS urls by implementing the Transport Adapter interface. This class will
usually be created by the :class:`Session <Session>` class under the
covers.
:param pool_connections: The number of urllib3 connection pools to cache.
:param pool_maxsize: The maximum number of connections to save in the pool.
:param int max_retries: The maximum number of retries each connection
should attempt. Note, this applies only to failed DNS lookups, socket
connections and connection timeouts, never to requests where data has
made it to the server. By default, Requests does not retry failed
connections. If you need granular control over the conditions under
which we retry a request, import urllib3's ``Retry`` class and pass
that instead.
:param pool_block: Whether the connection pool should block for connections.
Usage::
>>> import requests
>>> s = requests.Session()
>>> a = requests.adapters.HTTPAdapter(max_retries=3)
>>> s.mount('http://', a)
"""
__attrs__ = ['max_retries', 'config', '_pool_connections', '_pool_maxsize',
'_pool_block']
def __init__(self, pool_connections=DEFAULT_POOLSIZE,
pool_maxsize=DEFAULT_POOLSIZE, max_retries=DEFAULT_RETRIES,
pool_block=DEFAULT_POOLBLOCK):
if max_retries == DEFAULT_RETRIES:
self.max_retries = Retry(0, read=False)
else:
self.max_retries = Retry.from_int(max_retries)
self.config = {}
self.proxy_manager = {}
super(HTTPAdapter, self).__init__()
self._pool_connections = pool_connections
self._pool_maxsize = pool_maxsize
self._pool_block = pool_block
self.init_poolmanager(pool_connections, pool_maxsize, block=pool_block)
def __getstate__(self):
return dict((attr, getattr(self, attr, None)) for attr in
self.__attrs__)
def __setstate__(self, state):
# Can't handle by adding 'proxy_manager' to self.__attrs__ because
# self.poolmanager uses a lambda function, which isn't pickleable.
self.proxy_manager = {}
self.config = {}
for attr, value in state.items():
setattr(self, attr, value)
self.init_poolmanager(self._pool_connections, self._pool_maxsize,
block=self._pool_block)
def init_poolmanager(self, connections, maxsize, block=DEFAULT_POOLBLOCK, **pool_kwargs):
"""Initializes a urllib3 PoolManager.
This method should not be called from user code, and is only
exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param connections: The number of urllib3 connection pools to cache.
:param maxsize: The maximum number of connections to save in the pool.
:param block: Block when no free connections are available.
:param pool_kwargs: Extra keyword arguments used to initialize the Pool Manager.
"""
# save these values for pickling
self._pool_connections = connections
self._pool_maxsize = maxsize
self._pool_block = block
self.poolmanager = PoolManager(num_pools=connections, maxsize=maxsize,
block=block, strict=True, **pool_kwargs)
def proxy_manager_for(self, proxy, **proxy_kwargs):
"""Return urllib3 ProxyManager for the given proxy.
This method should not be called from user code, and is only
exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param proxy: The proxy to return a urllib3 ProxyManager for.
:param proxy_kwargs: Extra keyword arguments used to configure the Proxy Manager.
:returns: ProxyManager
"""
if not proxy in self.proxy_manager:
proxy_headers = self.proxy_headers(proxy)
self.proxy_manager[proxy] = proxy_from_url(
proxy,
proxy_headers=proxy_headers,
num_pools=self._pool_connections,
maxsize=self._pool_maxsize,
block=self._pool_block,
**proxy_kwargs)
return self.proxy_manager[proxy]
def cert_verify(self, conn, url, verify, cert):
"""Verify a SSL certificate. This method should not be called from user
code, and is only exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param conn: The urllib3 connection object associated with the cert.
:param url: The requested URL.
:param verify: Whether we should actually verify the certificate.
:param cert: The SSL certificate to verify.
"""
if url.lower().startswith('https') and verify:
cert_loc = None
# Allow self-specified cert location.
if verify is not True:
cert_loc = verify
if not cert_loc:
cert_loc = DEFAULT_CA_BUNDLE_PATH
if not cert_loc:
raise Exception("Could not find a suitable SSL CA certificate bundle.")
conn.cert_reqs = 'CERT_REQUIRED'
if not os.path.isdir(cert_loc):
conn.ca_certs = cert_loc
else:
conn.ca_cert_dir = cert_loc
else:
conn.cert_reqs = 'CERT_NONE'
conn.ca_certs = None
conn.ca_cert_dir = None
if cert:
if not isinstance(cert, basestring):
conn.cert_file = cert[0]
conn.key_file = cert[1]
else:
conn.cert_file = cert
def build_response(self, req, resp):
"""Builds a :class:`Response <requests.Response>` object from a urllib3
response. This should not be called from user code, and is only exposed
for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`
:param req: The :class:`PreparedRequest <PreparedRequest>` used to generate the response.
:param resp: The urllib3 response object.
"""
response = Response()
# Fallback to None if there's no status_code, for whatever reason.
response.status_code = getattr(resp, 'status', None)
# Make headers case-insensitive.
response.headers = CaseInsensitiveDict(getattr(resp, 'headers', {}))
# Set encoding.
response.encoding = get_encoding_from_headers(response.headers)
response.raw = resp
response.reason = response.raw.reason
if isinstance(req.url, bytes):
response.url = req.url.decode('utf-8')
else:
response.url = req.url
# Add new cookies from the server.
extract_cookies_to_jar(response.cookies, req, resp)
# Give the Response some context.
response.request = req
response.connection = self
return response
def get_connection(self, url, proxies=None):
"""Returns a urllib3 connection for the given URL. This should not be
called from user code, and is only exposed for use when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param url: The URL to connect to.
:param proxies: (optional) A Requests-style dictionary of proxies used on this request.
"""
proxy = select_proxy(url, proxies)
if proxy:
proxy = prepend_scheme_if_needed(proxy, 'http')
proxy_manager = self.proxy_manager_for(proxy)
conn = proxy_manager.connection_from_url(url)
else:
# Only scheme should be lower case
parsed = urlparse(url)
url = parsed.geturl()
conn = self.poolmanager.connection_from_url(url)
return conn
def close(self):
"""Disposes of any internal state.
Currently, this just closes the PoolManager, which closes pooled
connections.
"""
self.poolmanager.clear()
def request_url(self, request, proxies):
"""Obtain the url to use when making the final request.
If the message is being sent through a HTTP proxy, the full URL has to
be used. Otherwise, we should only use the path portion of the URL.
This should not be called from user code, and is only exposed for use
when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param proxies: A dictionary of schemes or schemes and hosts to proxy URLs.
"""
proxy = select_proxy(request.url, proxies)
scheme = urlparse(request.url).scheme
if proxy and scheme != 'https':
url = urldefragauth(request.url)
else:
url = request.path_url
return url
def add_headers(self, request, **kwargs):
"""Add any headers needed by the connection. As of v2.0 this does
nothing by default, but is left for overriding by users that subclass
the :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
This should not be called from user code, and is only exposed for use
when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param request: The :class:`PreparedRequest <PreparedRequest>` to add headers to.
:param kwargs: The keyword arguments from the call to send().
"""
pass
def proxy_headers(self, proxy):
"""Returns a dictionary of the headers to add to any request sent
through a proxy. This works with urllib3 magic to ensure that they are
correctly sent to the proxy, rather than in a tunnelled request if
CONNECT is being used.
This should not be called from user code, and is only exposed for use
when subclassing the
:class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
:param proxies: The url of the proxy being used for this request.
"""
headers = {}
username, password = get_auth_from_url(proxy)
if username and password:
headers['Proxy-Authorization'] = _basic_auth_str(username,
password)
return headers
def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple
:param verify: (optional) Whether to verify SSL certificates.
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
"""
conn = self.get_connection(request.url, proxies)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(request)
chunked = not (request.body is None or 'Content-Length' in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError as e:
# this may raise a string formatting error.
err = ("Invalid timeout {0}. Pass a (connect, read) "
"timeout tuple, or a single float to set "
"both timeouts to the same value".format(timeout))
raise ValueError(err)
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout
)
# Send the request.
else:
if hasattr(conn, 'proxy_pool'):
conn = conn.proxy_pool
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
low_conn.putrequest(request.method,
url,
skip_accept_encoding=True)
for header, value in request.headers.items():
low_conn.putheader(header, value)
low_conn.endheaders()
for i in request.body:
low_conn.send(hex(len(i))[2:].encode('utf-8'))
low_conn.send(b'\r\n')
low_conn.send(i)
low_conn.send(b'\r\n')
low_conn.send(b'0\r\n\r\n')
# Receive the response from the server
try:
# For Python 2.7+ versions, use buffering of HTTP
# responses
r = low_conn.getresponse(buffering=True)
except TypeError:
# For compatibility with Python 2.6 versions and back
r = low_conn.getresponse()
resp = HTTPResponse.from_httplib(
r,
pool=conn,
connection=low_conn,
preload_content=False,
decode_content=False
)
except:
# If we hit any problems here, clean up the connection.
# Then, reraise so that we can handle the actual exception.
low_conn.close()
raise
except (ProtocolError, socket.error) as err:
raise ConnectionError(err, request=request)
except MaxRetryError as e:
if isinstance(e.reason, ConnectTimeoutError):
# TODO: Remove this in 3.0.0: see #2811
if not isinstance(e.reason, NewConnectionError):
raise ConnectTimeout(e, request=request)
if isinstance(e.reason, ResponseError):
raise RetryError(e, request=request)
raise ConnectionError(e, request=request)
except ClosedPoolError as e:
raise ConnectionError(e, request=request)
except _ProxyError as e:
raise ProxyError(e)
except (_SSLError, _HTTPError) as e:
if isinstance(e, _SSLError):
raise SSLError(e, request=request)
elif isinstance(e, ReadTimeoutError):
raise ReadTimeout(e, request=request)
else:
raise
return self.build_response(request, resp)