Data scientists, data engineers, and application developers now have better programmability thanks to new updates from Snowflake, a provider of data clouds.
This week at its yearly user conference, Snowflake Summit 2022, in Las Vegas, the company made the update public.
With the release of Snowpark for Python, which is currently in public preview, and a native integration with Streamlit for quick application development and iteration, both of which are currently under development, Snowflake’s most recent innovations put Python in the spotlight. Along with making data stored in open formats and on-premises accessible in the Data Cloud, Snowflake is also streamlining access to more data with new improvements for working with streaming data.
These improvements make it simpler for data professionals and developers to build and collaborate with data quickly while utilizing Snowflake’s platform’s speed, simplicity, consistent governance, and security.
Increasing Python’s Use in Machine Learning and Application Development
These improvements make it simpler for data professionals and developers to build and collaborate with data quickly while utilizing Snowflake’s platform’s speed, simplicity, consistent governance, and security.
Data scientists, data engineers, and application developers now have access to a rich programming environment with Snowpark, the developer framework for Snowflake, allowing them to create scalable pipelines, applications, and machine learning (ML) workflows directly within Snowflake using their preferred languages and libraries. By facilitating seamless access to Python’s rich ecosystem of open-source packages and libraries in the Data Cloud, Snowflake is expanding what users can create with Snowpark for Python.
Snowpark for Python runs on the same Snowflake compute infrastructure as Snowflake pipelines and applications written in other languages thanks to a highly secure Python sandbox. As a result, developers can expect the same scalability, elasticity, security, and compliance benefits from Snowpark for Python as they have come to expect from Snowflake. Developers now have the exceptional chance to consolidate their Python-based data processing in Snowflake using Snowpark, streamlining and modernizing their data processing architecture.
Along with Snowpark for Python, other updates include:
- With the help of Python and Snowpark’s DataFrame APIs for Python, users can create pipelines, machine learning models, and applications directly in Snowsight, the Snowflake user interface. Development is sped up by code auto-complete and the quick productization of custom logic.
- With the help of Snowflake’s Streamlit Integration, which is still under development, users will be able to build interactive applications, securely share them with business teams to iterate, and work together to increase the impact of development.
- The currently under development Large Memory Warehouses gives users the ability to safely carry out memory-intensive operations, like feature engineering and model training on sizable datasets, using well-liked Python open-source libraries accessible through the Anaconda integration.
- SQL Machine Learning gives SQL users the ability to incorporate ML-powered predictions into their routine business intelligence and analytics to increase decision quality and speed, starting with time-series forecasting, which is currently in private preview.
Python is a well-liked choice among developers due to its robust syntax and extensive ecosystem of open-source packages. Thanks to Snowflake’s ongoing partnership with Anaconda, more Python packages are now seamlessly accessible in Snowflake, and all code is run in a highly secure sandboxed environment. As a result of Snowflake’s Python developments, the Snowpark Accelerated program has also continued to expand, with more partners using Python to increase the Data Cloud’s functionality in their preferred language.
In order to support machine learning (ML) and artificial intelligence (AI) solutions that make use of data in the Allegis Enterprise Data Platform on Snowflake, Allegis Group, a global talent solutions company, depends on Snowpark.
Joe Nolte, AI & MDM Architect, Allegis Group, said: “At its core, Snowpark is all about extensibility, and Snowpark for Python provides us with the tools we need to work with data effectively in our programming language of choice.”
“Snowpark is becoming our preferred framework for data science and application development, providing our teams with a seamless experience to easily collaborate with data and bring everyone onto the same platform for accelerated time-to-value.”
For developers to work more productively, to create more accurate ML models, and to offer more potent applications, they need quick and easy access to the right data. The upgrades to Snowflake let teams to experiment more quickly and with access to more data, resulting in improved programming capabilities and more insightful user experiences.
New innovations include:
- With Snowpipe Streaming, which is currently in private preview and allows for the serverless ingestion of streaming data, and Materialized Tables, which are currently under development and make declarative transformation of streaming data simple, Streaming Data Support aims to do away with the distinctions between streaming and batch pipelines.
- The currently under development Iceberg Tables in Snowflake, which will allow users to work with Apache Iceberg, a well-liked open table format, in external storage while utilizing the platform’s simplicity, performance, and consistent governance. This will streamline overall data management and increase architectural flexibility.
- With Snowflake’s External Tables for On-Premises Storage, customers can access their data in on-premises storage systems like Dell Technologies, Pure Storage, and others to take use of the Data Cloud’s elasticity without relocating that data. This feature is currently in private preview.
Christian Kleinerman, senior VP of product, Snowflake, said: “We are heavily investing in Python to make it easier for data scientists, data engineers, and application developers to build even more in the Data Cloud, without governance trade-offs.
“Our latest innovations extend the value of our customers’ data-driven ecosystems, enabling them with more access to data and new ways to develop with it directly in Snowflake. These capabilities, paired with Snowflake’s best of class data security and privacy, are changing the way teams experiment, iterate, and collaborate with data to drive value.”
Article published on June 16, 2022
If you like this article, please share it:
|