logo
down
shadow

Python: Have an iterator iterate over log messages on a different thread


Python: Have an iterator iterate over log messages on a different thread

By : jake talledo
Date : November 17 2020, 01:00 AM
hope this fix your issue I have 2 threads running: , Well, after some playing around, this is the solution I came up with:
code :
def log_iterator(logger_names):
    '''
    Get an iterator that returns LogRecord objects whenever a log message is made.

    logger_names can be:
        a string identifying which logger you want to get messages iterate from
        a list identifying all the loggers you want to get messages from
        a dict<name: level> where level identifies the minimum loggling level 
            you want (default is INFO)
    '''
    if isinstance(logger_names, str):
        logger_names = {logger_names: logging.INFO}
    elif isinstance(logger_names, (list, tuple)):
        logger_names = {name: logging.INFO for name in logger_names}

    log_message_event = threading.Event()
    for logger_name, log_level in logger_names.iteritems():
        logger = logging.getLogger(logger_name)
        logger.addHandler(FireOnLog(log_message_event, log_level))

    while True:
        log_message_event.wait()
        record = log_message_event.record
        log_message_event.clear()
        yield record


class FireOnLog(logging.StreamHandler):
    '''
    Custom loghandler that fires an event every time a log message 
    comes in.  To use, you need to add this handler to a logger.
    '''

    def __init__(self, event, log_level = logging.INFO):
        '''
        Event is a threading.Event object
        log_level is the minimum logging level you want
        '''
        logging.StreamHandler.__init__(self)
        self._event = event
        self._log_level = log_level

    def emit(self, record):
        if record.levelno >= self._log_level:
            self._event.record = record
            self._event.set()
def _print_messages():
    print 'Started Printing loop'
    for log_record in log_iterator(['testlog', 'anotherlog']):
        print '%s:%s:%s' % (log_record.levelname, log_record.name, log_record.message)


def test_log_handling():

    t = threading.Thread(target = _print_messages)
    t.start()

    time.sleep(.1)
    logging.getLogger('testlog').critical('Message!')
    logging.getLogger('anotherlog').critical('Another Message right away')
    logging.getLogger('badlog').critical("Someething useless that you don't want to hear")
    time.sleep(.1)
    logging.getLogger('testlog').critical('Another Message after some wait')


Share : facebook icon twitter icon
In Python, is it possible to iterate over values stored in a thread local by each thread?

In Python, is it possible to iterate over values stored in a thread local by each thread?


By : user3787403
Date : March 29 2020, 07:55 AM
seems to work fine Not in Python, although you might be able to dig around in the interpreter state in C and acquire this information (although I doubt it). It would be reasonable to assume that this might be possible if local() returned a singleton for each thread, but it does not - calling local() multiple times in the same thread will generate multiple unique thread-local storage instances (not return a reference to the same object) and as such there is not a single field in the Python internal thread structure that holds this data, making it difficult to inspect.
Iterate an iterator by chunks (of n) in Python?

Iterate an iterator by chunks (of n) in Python?


By : Colin Anthony
Date : March 29 2020, 07:55 AM
this will help The grouper() recipe from the itertools documentation's recipes comes close to what you want:
code :
def grouper(n, iterable, fillvalue=None):
    "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return izip_longest(fillvalue=fillvalue, *args)
[my_list[i:i + chunk_size] for i in range(0, len(my_list), chunk_size)]
def grouper(n, iterable):
    it = iter(iterable)
    while True:
       chunk = tuple(itertools.islice(it, n))
       if not chunk:
           return
       yield chunk
In Python, how do I iterate over one iterator and then another?

In Python, how do I iterate over one iterator and then another?


By : Peter Kensho
Date : March 29 2020, 07:55 AM
help you fix your problem Use itertools.chain:
code :
from itertools import chain
for line in chain(file1, file2):
   pass
import fileinput
for line in fileinput.input(['file1', 'file2']):
   pass
iterate over a list as iterator in python

iterate over a list as iterator in python


By : user3339780
Date : March 29 2020, 07:55 AM
I hope this helps you . If I understood right, you're looking for generators:
code :
def mylist():
    for i in range(5):
        elem = myobject(i)
        yield elem
class myobject:
    def __init__(self, i):
        self.text = 'hello ' + str(i)

def mylist():
    for i in range(5):
        elem = myobject(i)
        yield elem

for obj in mylist():
    print obj.text
Python -- Iterate an iterator twice

Python -- Iterate an iterator twice


By : glowie
Date : March 29 2020, 07:55 AM
I think the issue was by ths following , You could use itertools.tee to create multiple copies of the generator
code :
from itertools import tee

def MyIter():
    yield 1
    yield 2
    yield 3
    yield 4

def PrintCombos(x):
    it1, it2 = tee(x, 2)
    for a in it1:
        it2, it3 = tee(it2, 2)
        for b in it3:
        print("{0}-{1}".format(a, b))

PrintCombos(MyIter())
Related Posts Related Posts :
  • pretty printing numpy ndarrays using unicode characters
  • Frequent pattern mining in Python
  • How can I make a set of functions that can be used synchronously as well as asynchronously?
  • Convert one dice roll to two dice roll
  • count occourrence in a list
  • Writing an If condition to filter out the first word
  • to read file and compare column in python
  • Install python-numpy in the Virtualenv environment
  • `.select_by_visible_text()` is failed to select element?
  • Unable to send data multiple requests in a single connection — socket error
  • Pandas HDFStore unload dataframe from memory
  • Creating a custom admin view
  • How do you get the user role of the currently logged in user in Ckan?
  • Speed up Numpy Meshgrid Command
  • Python error - name lengths
  • appending text to a global variable
  • Python Mistake - Number of letters in name
  • Searching for a sequence in a text
  • Testing logging output with pytest
  • How do I change my default working directory for Python (Anaconda) on VSCode?
  • .lower() for x in list, not working, but works in another scenario
  • Program gives error "List indices must not be string"
  • pyqt: Memory Usage
  • Confused about classes in Learn Python the Hard Way ex43?
  • Extracting unrecognized information from many CSV files
  • How do I connect to Postgresql server from Python?
  • Append rows to a pandas DataFrame without making a new copy
  • Scrapy: Importing a package from the project that's not in the same directory
  • launching Excel application using Python to view the CSV file , but CSV file is opening in read mode and cant view the d
  • Making a list in user-defined functions
  • Pyserial microcontroller to host communication
  • Plotting a line in between subplots
  • function not returning value. Error "NameError: name 'urlss' is not defined"
  • How to perform cartesian product with Tensorflow?
  • Multiple independent random number streams from single seed
  • I Need a simple and short python3 code that count secounds in a background process
  • No module named constants
  • from django 1.4 to django 1.5- argument 'verify_exists' what s replacement?
  • Slash replacement inside a raw string
  • Reordering columns/rows of a pivot_table?
  • MySQLdb.cursors.Cursor.execute does not work
  • Python module being reimported when imported at different places
  • Is the Session object from Python's Requests library thread safe?
  • Python Regex: Finding First and Last Names
  • Order by selection in List view of OpenERP 7.0
  • Reading input values in ipython notebook
  • List of dictionaries - how to read a specific value in a dictionary
  • writing os.system output to file
  • Create dictionary from points list and multiple attribute lists
  • How to write a table line by line with for loop
  • Map projection and forced interpolation
  • Django FBV's "render_to_response" equivalent in Class-Based-View?
  • Paramiko raises "SFTPError: Garbage packet received"
  • python pandas operations on columns
  • python list appending is not working
  • Speeding up matplotlib scatter plots
  • For each element of the list find closest date from a different list
  • How to prepend new rows at the beginning of an existing csv file?
  • how to make database robust to process kills with sqlite postgress and sqlalchemy?
  • finding a set of ranges that a number fall in
  • shadow
    Privacy Policy - Terms - Contact Us © ourworld-yourmove.org