Build and operationalize open-source State-of-the-Art models in Azure Machine Learning
Recent advancements in AI propelled the rise of large foundation models that are trained on a vast quantity of data and can be easily adapted to a wide variety of applications across various industries. This emerging trend provides a unique opportunity for enterprises to build and use foundation models in their deep learning workloads.
Microsoft announcing the upcoming public preview of foundation models in Azure Machine Learning. It provides Azure Machine Learning with native capabilities that enable customers to build and operationalize open-source foundation models at scale. With these new capabilities, organizations will get access to curated environments and Azure AI Infrastructure without having to manually manage and optimize dependencies. Azure Machine learning professionals can easily start their data science tasks to fine-tune and deploy foundation models from multiple open-source repositories, starting from Hugging Face, using Azure Machine Learning components and pipelines. This service will provide you with a comprehensive repository of popular open-source models for multiple tasks like natural language processing, vision, and multi-modality through the Azure Machine Learning built in registry.
The Next Generation of Azure Cognitive Services for Vision.
Today, Azure Cognitive Services for Vision released its next generation of capabilities powered by the Florence large foundational model. This new Microsoft model delivers significant improvements to image captioning and groundbreaking customization capabilities with few-shot learning. Until today, model customization required large datasets with hundreds of images per label to achieve production quality for vision tasks. But, Florence is trained on billions of text-image pairs, allowing custom models to achieve high quality with just a few images. This lowers the hurdle for creating models that can fit challenging use cases where training data is limited.
Users can try the new capabilities of Vision underpinned by the Florence model through Vision Studio. This tool demonstrates a full set of prebuilt vision tasks, including automatic captioning, smart cropping, classifying images and a summarizing video with natural language, and much more. Users can also see how the tool helps track movements, analyze environments, and provide real-time alerts.
New Responsible AI Toolbox Additions
The Responsible AI Mitigations Library allows practitioners to experiment with different mitigation techniques more easily, while the Responsible AI Tracker uses visualizations to demonstrate the effectiveness of different mitigations for more informed decision-making. The new mitigations library bolsters mitigation by offering a means of managing failures that occur in data preprocessing. The library complements the toolbox’s Fairlearn fairness assessment tool, which focuses on mitigations applied during training time. The tracker allows practitioners to look at performance for subsets of data across iterations of a model to help them determine the most appropriate model for deployment. When used with other tools in the Responsible AI Toolbox, they offer a more efficient and effective means to help improve the performance of systems across users and conditions. These tools are made open source on GitHub and integrated into Azure Machine Learning.
Accelerate large-scale AI with Azure AI infrastructure
Azure AI Infrastructure provides massive scale-up and scale-out capabilities for the most advanced AI workloads in the world. This is a key factor as to why leading AI companies, including our partners at OpenAI continue to choose Azure to advance their AI innovation on Azure AI. Our results for training OpenAI’s GPT-3 on Azure AI Infrastructure using Azure NDm A100 v4 virtual machines with NVIDIA’s open-source framework, NVIDIA NeMo Megatron, delivered a 530B-parameter benchmark on 175 virtual machines, resulting in a scalability factor of 95 percent. When Azure AI infrastructure is used together with a managed end-to-end machine learning platform, such as Azure Machine Learning, it provides the vast compute needed to enable organizations to streamline management and orchestration of large AI models and help bring them into production.
MLflow 2.0 and Azure Machine Learning
MLflow is an open-source platform for the complete machine learning lifecycle, from experimentation to deployment. Being one of the MLflow contributors, Azure Machine Learning made its workspaces MLflow-compatible, which means organizations can use Azure Machine Learning workspaces in the same way that they use an MLflow tracking server. MLflow has recently released its new version, MLflow 2.0, which incorporates a refresh of the core platform APIs based on extensive feedback from MLflow users and customers, which simplifies the platform experience for data science and machine learning operations workflows. We’re excited to announce that MLflow 2.0 is also supported in Azure Machine Learning workspaces..
Azure AI is empowering developers and organizations to build cutting-edge AI solutions with its rich set of open-source technologies. From leveraging pre-trained models to customizing AI capabilities with new technologies like Hugging Face foundation models, to integrating responsible AI practices with new open-source tools, Azure AI is driving innovation and efficiency in the AI industry. With Azure AI infrastructure, organizations can accelerate their large-scale AI workloads and achieve even greater results. Take a deep dive into what open-source projects and features Microsoft’ve announced at Azure Open Source Day 2023.
(Resource by Microsoft)
You want to learn more about Azure AI, let ITM help you.
