AWS Lake Formation extends table permissions to access underlying data in Amazon S3

AWS has enhanced Lake Formation to enable direct access to underlying data files stored in Amazon S3 for tables registered in the AWS Glue Data Catalog. The new capability allows users to manage both SQL queries and direct file access through a single set of Lake Formation table permissions, eliminating the need for separate permission management systems. Lake Formation now provides temporary, scoped credentials for registered S3 locations based on existing table permissions, with SELECT permissions granting read access and SUPER permissions providing both read and write capabilities. The feature comes integrated with Amazon EMR 7.13 and later versions, enabling data engineers to access data files directly from Spark jobs for tasks like model training, feature engineering, and debugging data quality issues. Organizations can also integrate Apache Spark or Trino applications through AWS-provided APIs or an open source plugin. All access operations are automatically logged in AWS CloudTrail, creating a unified audit trail across both SQL and file-based operations on tables. The enhancement is available at no additional cost across all AWS regions where Lake Formation operates, addressing a common challenge in data lake architectures where teams previously needed to manage separate permission systems for database-level and file-level access to the same underlying data.