Data Science Wire

Formal Support for Windows?

Reddit r/apachespark1w4 min read

Do the Apache Spark maintainers formally support windows for production workloads? I understand that the majority of this platform is JVM-based and the support for Windows is based on the compatibility of the JVM itself. However there are some peripheral features that seem to be disallowed on Windows - even when running it on a fully functional JVM. When working with the python ecosystem, I often find libraries that are particularly incompatible with windows. In the same way I've found that Apache's pyspark extensions on Windows are lackluster (eg. things like "spark.python.use.daemon" are dis

Read the full story at Reddit r/apachespark

More in Data Engineering