What are the issues one could face with using SparkListener

I am thinking of adding SparkListener to our Spark Cluster and then get metrics and query plan for every job. This will be loaded to a database in Background thread in the driver. Are there any issues doing the above? Like will Driver face any performance issues or OOM because of this? It will be a simple listener without any heavy work. Just collecting metrics and feeding to DB. submitted by /u/inglocines [link] [comments]