-> this happens if the processing of items is slower as the fetching of
new
-> if a big library is synced, the old behavior could lead to extensive
use of memory
-> the semaphore acts like a buffer that only allows fetching of new
items from the library if old ones are processed
-> the current size of the 'buffer' is hard coded to 2 * [max. item
fetch limit] * [number of download threads]
Added ThreadPoolExecutor and used to process GET requests in multiple
threads which enables chunks of data to always be available for
processing. Processing of the data can happen as soon as the first chunk
arrives.
Refactored the code to help implement. The idea is the "params" are
built in batch and passed to the thread pool which get the actual
results.
