sbws.core package

Submodules

sbws.core.cleanup module

Util functions to cleanup disk space.

sbws.core.cleanup.gen_parser(sub)

Helper function for the broader argument parser generating code that adds in all the possible command line arguments for the cleanup command.

Parameters:sub (argparse._SubParsersAction) – what to add a sub-parser to

sbws.core.cleanup.main(args, conf)

Main entry point in to the cleanup command.

Parameters:

• args (argparse.Namespace) – command line arguments
• conf (configparser.ConfigParser) – parsed config files

sbws.core.generate module

sbws.core.generate.gen_parser(sub)

sbws.core.generate.main(args, conf)

sbws.core.scanner module

Measure the relays.

sbws.core.scanner.create_path_relay(relay, dest, rl, cb, relay_as_entry=True)

sbws.core.scanner.dispatch_worker_thread(*a, **kw)

sbws.core.scanner.dumpstacks()

sbws.core.scanner.error_no_circuit(circ_fps, nicknames, reason, relay, dest, our_nick)

sbws.core.scanner.error_no_helper(relay, dest, our_nick='')

sbws.core.scanner.force_get_results(pending_results)

Try to get either the result or an exception, which gets logged.

It is call by wait_for_results() when the time waiting for the results was long.

To get either the Result or an exception, call get() with timeout. Timeout is low since we already waited.

get is not call before, because it blocks and the callbacks are not call.

sbws.core.scanner.gen_parser(sub)

sbws.core.scanner.get_random_range_string(content_length, size)

Return a random range of bytes of length size. content_length is the size of the file we will be requesting a range of bytes from.

For example, for content_length of 100 and size 10, this function will return one of the following: ‘0-9’, ‘1-10’, ‘2-11’, […] ‘89-98’, ‘90-99’

sbws.core.scanner.main(args, conf)

sbws.core.scanner.main_loop(args, conf, controller, relay_list, circuit_builder, result_dump, relay_prioritizer, destinations, pool)

Starts and reuse the threads that measure the relays forever.

It starts a loop that will be run while there is not and event signaling that sbws is stopping (because of SIGTERM or SIGINT).

Then, it starts a second loop with an ordered list (generator) of relays to measure that might a subset of all the current relays in the Network.

For every relay, it starts a new thread which runs measure_relay to measure the relay until there are max_pending_results threads. After that, it will reuse a thread that has finished for every relay to measure. It is the the pool method apply_async which starts or reuse a thread. This method returns an ApplyResult immediately, which has a ready methods that tells whether the thread has finished or not.

When the thread finish, ie. ApplyResult is ready, it triggers result_putter callback, which put the Result in ResultDump queue and complete immediately.

ResultDump thread (started before and out of this function) will get the Result from the queue and write it to disk, so this doesn’t block the measurement threads.

If there was an exception not catched by measure_relay, it will call instead result_putter_error, which logs the error and complete immediately.

Before the outer loop iterates, it waits (non blocking) that all the Results are ready calling wait_for_results. This avoid to start measuring the same relay which might still being measured.

sbws.core.scanner.measure_bandwidth_to_server(session, conf, dest, content_length)

Returns tuple:results or None if the if the measurement fail. None or exception if the measurement fail.

sbws.core.scanner.measure_relay(args, conf, destinations, cb, rl, relay)

Select a Web server, a relay to build the circuit, build the circuit and measure the bandwidth of the given relay.

Return Result:a measurement Result object

sbws.core.scanner.measure_rtt_to_server(session, conf, dest, content_length)

Make multiple end-to-end RTT measurements by making small HTTP requests over a circuit + stream that should already exist, persist, and not need rebuilding. If something goes wrong and not all of the RTT measurements can be made, return None. Otherwise return a list of the RTTs (in seconds).

Returns tuple:results or None if the if the measurement fail. None or exception if the measurement fail.

sbws.core.scanner.result_putter(result_dump)

Create a function that takes a single argument – the measurement result – and return that function so it can be used by someone else

sbws.core.scanner.result_putter_error(target)

Create a function that takes a single argument – an error from a measurement – and return that function so it can be used by someone else

sbws.core.scanner.run_speedtest(args, conf)

Initializes all the data and threads needed to measure the relays.

It launches or connect to Tor in a thread. It initializes the list of relays seen in the Tor network. It starts a thread to read the previous measurements and wait for new measurements to write them to the disk. It initializes a class that will be used to order the relays depending on their measurements age. It initializes the list of destinations that will be used for the measurements. It initializes the thread pool that will launch the measurement threads. The pool starts 3 other threads that are not the measurement (worker) threads. Finally, it calls the function that will manage the measurement threads.

sbws.core.scanner.stop_threads(signal, frame, exit_code=0)

sbws.core.scanner.timed_recv_from_server(session, dest, byte_range)

Request the byte_range from the URL at dest. If successful, return True and the time it took to download. Otherwise return False and an exception.

sbws.core.scanner.wait_for_results(num_relays_to_measure, pending_results)

Wait for the pool to finish and log progress.

While there are relays being measured, just log the progress and sleep TIMEOUT_MEASUREMENTS (3mins), which is aproximately the time it can take to measure a relay in the worst case.

When there has not been any relay measured in TIMEOUT_MEASUREMENTS and there are still relays pending to be measured, it means there is no progress and call force_get_results().

This can happen in the case of a bug that makes either measure_relay(), result_putter() (callback) and/or result_putter_error() (callback error) stall.

Note

in a future refactor, this could be simpler by:

1. Initializing the pool at the begingging of each loop
2. Callling close(); join() after apply_async(), to ensure no new jobs are added until the pool has finished with all the ones in the queue.

As currently, there would be still two cases when the pool could stall:

1. There’s an exception in measure_relay and another in callback_err
2. There’s an exception callback.

This could also be simpler by not having callback and callback error in apply_async and instead just calling callback with the pending_results.

(callback could be also simpler by not having a thread and queue and just storing to disk, since the time to write to disk is way smaller than the time to request over the network.)

sbws.core.stats module

sbws.core.stats.gen_parser(sub)

Helper function for the broader argument parser generating code that adds in all the possible command line arguments for the stats command.

Parameters:sub (argparse._SubParsersAction) – what to add a sub-parser to

sbws.core.stats.main(args, conf)

Main entry point into the stats command.

Parameters:

• args (argparse.Namespace) – command line arguments
• conf (configparser.ConfigParser) – parsed config files

sbws.core.stats.print_stats(args, data)

Called from main to print various statistics about the organized data to stdout.

Parameters:

• args (argparse.Namespace) – command line arguments
• data (dict) – keyed by relay fingerprint, and with values of sbws.lib.resultdump.Result subclasses

Module contents