Big Data & AI Buzzwording – What are some practical on-premises data tools for NFS shares?
Reddit r/dataengineering15h4 min read
AI data catalog, data warehouse, data lake, data index, .... It feels like all search results for indexing and cataloging are focused on structured data in cloud environments. But what if you walk into an environment with petabyte-scale NFS storage, full of measurement data? Users copy this data across different shares for various purposes (archiving, HPC computing, etc.). There is no overview of what data exists or where duplicates are located. And when someone needs to find data, they have to dig through directories or ask others to send them links. It feels like this is an industry-independ