Data preparation strategies in Kubeflow for cloud-native AI systems

Abstract

This article presents the main findings from an in-depth study of data preparation strategies using Kubeflow in cloud-native AI systems deployed on Azure Kubernetes Service. The results demonstrate that integrating Kubeflow Pipelines with Azure-native tools enables scalable and automated processing of large datasets, significantly improving training efficiency and model accuracy. The use of TensorFlow Data Validation proved effective in detecting schema anomalies and data drift, enhancing data reliability across iterative ML workflows. A case study confirms that the implemented pipeline reduced data processing time by 35 % and increased pipeline reproducibility through integrated metadata tracking and data versioning. These outcomes highlight Kubeflow’s practical value in supporting efficient, traceable, and production-ready AI pipelines in enterprise-grade cloud environments.