Logging from a Publish plugin?

Pipeline Integrations
1

Hey all - I’m trying to troubleshoot some exceptions that are happening in a collector during publish from Flame. For some reason the shot entity is not picking up either a name or a code attribute.
I’m trying to do any sort of logging from a custom collector hook. The documentation indicates that the publish API commandeers all logging output, and assembles it in the UI. Unfortunately, the UI does not seem to display any logging output from the collector.
I even tried writing out to a simple text file, which has no data in it.
Is there any way to get output from a collector hook?

2

Do you use self.logger, or self.parent.logger? Not sure but one of these two should give you the log in the publish dialog.
Otherwise, you could just watch the log file that the output goes to, whichever it is.

Another option you have is remote debugging the hook e.g. with rempte_pdb - you set up a breakpoint in the hook and connect to it using telnet. This way you can examine the complete state of the program at the point you need.

3

@mmoshev that remote_pdb suggestion is genius. Thanks so much, I’ll give that a try!

4

I should also say that both self.logger and self.parent.logger are overridden by the publish engine - all the logging output is sucked into the GUI. Any output from collector hooks feels like it just gets lost.

I ended up putting this in the constructor for the hook:

        self.debug_log = False
        if self.debug_log:
            manager = logging.root.manager
            manager.disabled = logging.NOTSET
            for logger in manager.loggerDict.values():
                if isinstance(logger, logging.Logger):
                    logger.setLevel(logging.NOTSET)
                    logger.propagate = True
                    logger.disabled = False
                    logger.filters.clear()
                    handlers = logger.handlers.copy()
                    for handler in handlers:
                        try:
                            handler.acquire()
                            handler.flush()
                            handler.close()
                        except (OSError, ValueError):
                            pass
                        finally:
                            handler.release()
                        logger.removeHandler(handler)
            self._logger = logging.getLogger("collector_hook")
            self._logger.setLevel(logging.DEBUG)
            handler = logging.FileHandler("/tmp/collector_hook.log")
            formatter = logging.Formatter('%(asctime)s [%(process)d %(levelname) %(name)s] %(message)s')
            handler.setFormatter(formatter)
            self._logger.addHandler(handler)
        else:
            self._logger = self.engine.logger

This hijacks control of the logger away from the engine, which lets you redirect the output to /tmp/collector_hook.log. Kind of a kludge but it did the trick.

1 Like
5

Hello! You can also take a look at this threat with good suggestions for remote debugging: Remote Debugging