Directory Downloader

class bushel.downloader.DirectoryDownloader(initial_consensus=None, directory_cache_mode=None, max_concurrency=9)

The DirectoryDownloader provides an asyncio-compatible wrapper around the stem DescriptorDownloader, with two modes of operation:

• Directory Cache ([dir-spec] §4)

• Client ([dir-spec] §5)

The DirectoryDownloader will not initiate downloads on its own intiative. It must be driven to perform downloads through the methods provided.

Note

As a valid consensus is required to implement parts of the functionality, the latest consensus is cached internally. This cached consensus should not be relied upon by external code. The cached consensus will never be served as a response to a request for a consensus.

directory_authorities()

Returns a list containing either a DirPort or an ORPort for each of the directory authorities.

directory_caches(extra_info=False)

Returns a list containing either a DirPort or an ORPort for each of the directory caches known from the latest consensus. If no consensus is known, this will return authorities() instead.

Parameters

extra_info (bool) – Whether the list returned should contain only directory caches that cache extra-info descriptors.

relay_extra_info_descriptors(digests, published_hint=None)

Retrieves multiple extra-info descriptors from directory servers.

Parameters

• digests (list(str)) – Hex-encoded digests for the descriptors.

• published_hint (datetime) – Provides a hint on the published time. Currently this is unused, but is accepted for compatibility with other directory sources. In the future this may be used to avoid attempts to download descriptors that it is likely are long gone.

Returns

A list of stem.descriptor.extrainfo_descriptor.RelayExtraInfoDescriptor.

relay_microdescriptors(microdescriptor_hashes, valid_after_hint=None)

Retrieves multiple server descriptors from directory servers.

Parameters

• hashes (list(str)) – base64-encoded hashes for the microdescriptors.

• valid_after_hint (datetime) – Provides a hint on the valid_after time. Currently this is unused, but is accepted for compatibility with other directory sources. In the future this may be used to avoid attempts to download descriptors that it is likely are long gone.

Returns

A list of stem.descriptor.microdescriptor.Microdescriptor.

relay_server_descriptors(digests, published_hint=None)

Retrieves multiple server descriptors from directory servers.

Parameters

• digests (list(str)) – Hex-encoded digests for the descriptors.

• published_hint (datetime) – Provides a hint on the published time. Currently this is unused, but is accepted for compatibility with other directory sources. In the future this may be used to avoid attempts to download descriptors that it is likely are long gone.

Returns

A list of stem.descriptor.server_descriptor.RelayDescriptor.

bushel.downloader.relay_extra_info_descriptors_query_path(digests)

Generates a query path to request extra-info descriptors by digests from a directory server. For example:

>>> digests = ["A94A07B201598D847105AE5FCD5BC3AB10124389",
...            "B38974987323394795879383ABEF4893BD4895A8"]
>>> relay_extra_info_descriptors_query_path(digests)  # doctest: +ELLIPSIS
'/tor/extra/d/A94A07B201598D847105...24389+B3897498732339479587...95A8'

These query paths are defined in appendix B of [dir-spec]. By convention, these URLs use upper-case hex-encoded SHA-1 digests and so this function will ensure that digests are upper-case. Directory server implementations should not rely on this behaviour.

Parameters

digests (list(str)) – The hex-encoded SHA-1 digests for the descriptors.

Returns

Query path as a str.

bushel.downloader.relay_microdescriptors_query_path(microdescriptor_hashes)

Generates a query path to request microdescriptors by their hashes from a directory server. For example:

>>> microdescriptor_hashes = ["Z62HG1C9PLIVs8jLi1guO48rzPdcq6tFTLi5s27Zy4U",
...                           "FkiLuQJe/Gqp4xsHfheh+G42TSJ77AarHOGrjazj0Q0"]
>>> relay_microdescriptors_query_path(microdescriptor_hashes)  # doctest: +ELLIPSIS
'/tor/micro/d/Z62HG1C9PLIVs8jL...Li5s27Zy4U-FkiLuQJe/Gqp4xsHf...rjazj0Q0'

These query paths are defined in appendix B of [dir-spec].

Parameters

digests (list(str)) – The base64-encoded hashes for the descriptors.

Returns

Query path as a str.

bushel.downloader.relay_server_descriptors_query_path(digests)

Generates a query path to request server descriptors by digests from a directory server. For example:

>>> digests = ["A94A07B201598D847105AE5FCD5BC3AB10124389",
...            "B38974987323394795879383ABEF4893BD4895A8"]
>>> relay_server_descriptors_query_path(digests)  # doctest: +ELLIPSIS
'/tor/server/d/A94A07B201598D847105...24389+B3897498732339479587...95A8'

These query paths are defined in appendix B of [dir-spec]. By convention, these URLs use upper-case hex-encoded SHA-1 digests and so this function will ensure that digests are upper-case. Directory server implementations should not rely on this behaviour.

Parameters

digests (list(str)) – The hex-encoded SHA-1 digests for the descriptors.

Returns

Query path as a str.