My motivations, experiences and future plans for a deep learning education
As I reflect on my journey in machine learning in 2023, I am amazed at the progress and evolution that has occurred since my initial article. ChatGPT and Dalle-2 are bringing a new wave of creativity and innovation, and new AI tools are being released at an incredible rate.
Machine learning and deep learning knowledge is becoming increasingly vital for everyone, regardless of background. Now as a business innovation and design thinking coach, I believe that design thinking and the human-centred approach to innovation and growth will lead to the development of deep learning solutions that drive business growth and improve people's lives by making sure that deep learning is developed in a way that puts the human experience first. Let's work towards making this education accessible for all.
My first article covered the machine learning foundational work; learning Python, exploratory data analytics, machine learning practice and machine learning theory. It was time for me to move onto the next phase of the plan, having reached a good level of knowledge. Practice my machine learning skills and dive into deep learning.
This updated article covers my motivations, explanations of deep learning, recommended courses for theory and practice, cloud platforms, AI governance and AI ethics.
What motivated me to start a deep learning education?
I have always had a strong desire for continuous learning, and I saw deep learning as a new challenge and opportunity for growth.
I see deep learning & ai as a tool that can augment human capabilities, translating theory into practice to solve real problems with human-centred values.
At the beginning of my dive into a deep learning education, it was quite daunting to know where to begin. It’s a vast topic, and it’s easy to get bogged down and frustrated with the many topics and mathematics. The key was perseverance.
Deep learning is part of a broader family of machine learning methods based on artificial neural networks
What Is Deep Learning?
The definition of deep learning has led to many debates. Still, for now, “deep learning is part of a broader family of machine learning methods based on artificial neural networks”. (Wikipedia)
The explosion in deep learning over recent years has been made possible by three things, the vast amount of data available, the increase in computational power and algorithm innovation.
What are examples of Deep Learning?
Self Driving Cars, Object detection, decision making.
Object Classification, visual search, face recognition.
Natural language processing, spam filters, Siri, Alexa, or Google Assistant.
Large language models, ChatGPT. DALL-E 2.
Health Care. MRI scans, CT scans, records analysis.
What does a Neural Network look like?
Google has developed this playground where you can “tinker” with neural network in your browser. Check out the playground TensorFlow site.
What was my approach to learning deep learning?
My approach to deep learning was threefold, understand enough of the theory to interpret the algorithms, learn the coding and then practice them. This approach led me to the deeplearning.ai speciality (The theory), Fast.Ai (The coding) and Kaggle (The practice).
What does the deeplearning ai speciality on Coursera cover?
Andrew Ng’s original machine learning course was clear, well-structured and made the mathematics understandable. It gave me the confidence to tackle deep learning with Andrew Ng’s latest speciality.
The deeplearning ai speciality covers a deep neural network course, Convolutional neural networks (e.g. Image recognition, radiology imagery), Sequence models (e.g. speech recognition, chatbots, language processing), advice on how to improve your models (hyper-parameter tuning), structuring your deep learning projects and machine learning strategies.
The programming exercises are in Python and help clarify the content of the lectures. You will build a neural network, image classification, facial recognition, natural language processing and trigger word detection (e.g. hey Siri or Alexa). Building a neural network with mathematics as opposed to programming libraries provides a level of understanding that some of the more practical courses don’t cover. While this level of depth may not be required, and it is useful to learn what happens under the hood.
Over 16 weeks the five courses are layered out, but you can take them at your own pace. This specialisation teaches the most essential and foundational principles of deep learning. If you’re a strictly hands-on learner, then this course is not for you. The blend of theory in this course is complementary to the practical Fast.ai course.
What does the Fast.ai practical deep learning course offer?
Fast.ai Practical deep learning for Coders is a free, hands-on course that aims to teach people to code in Python, state of the art, deep learning techniques. Fast.ai aims to make deep learning simpler to use by getting more people from all backgrounds involved through free courses, research and a community. Quoting from the Fast.Ai website. “What I hope is that lots of people will realise that state-of-the-art results of deep learning are something they can achieve even if they’re not a Stanford University deep learning PhD.”
Fast.ai covers data cleaning, image classification, multi-label classification; Segmentation NLP; Tabular data; Collaborative filtering; Embeddings, Regularisation, data ethics and Residual networks.
There are no formal programming exercises or quizzes. Still, you are encouraged to make your projects from what you have learned in the lectures. Using the latest state of the art technology and the inbuilt Fast.ai libraries, you’re taught to code your models. The fast.ai libraries are built on top of open-source PyTorch library. After two lessons I had written my image classifier on AWS Sagemaker.
There is a strong community to help with your challenges and to critique your work. If you are a hands-on learner, this course is for you, even when techniques such as backpropagation are covered they are presented practically as opposed to the underlying mathematics.
How can Kaggle competitions help in learning deep learning?
Partway through the deep learning specialisation, I took a break and entered some machine learning competitions on Kaggle. Kaggle is a data science community with powerful tools and resources to help you achieve your data science goals.
Kaggle is a great way to test your skills. There is a leaderboard for each competition where individuals or teams compete against each other. Also, there are examples of winning code that helps develop your skills in this crucible of data science.
There are four beginners competitions. Data classification: predicting house prices. Image recognition using hand-written digits, and natural language processing analysing tweets for the sentiment. Jeremy Howard from Fast.ai, a Kaggle master himself, suggests that “if you’re any good you should get into the top 10% of a competition”.
My first score put me in the top 73%, a rude awakening after a long period of studying! After repeated attempts late into the night, I reached a level of scores that I was happy with, for now!
Introduction to machine learning classifications. Top 5% 749/16523
Advanced Regression Techniques predict sales prices Top 14% 605/4550
Computer vision hand-written digit recogniser Top 16% 335/2230
Natural Language processing predicting tweet outcomes. Top 44% 1472/3423
What can you learn from Kaggle?
· The importance of data preparation cannot be understated. Learning how to clean the data, feature engineering, transformations, randomisation, train/test split have a massive impact on your outputs.
· How do you choose and train the right model? Different algorithms are for different tasks.
· The skills to evaluate and tune the model. Advice on selecting the right metrics and the art form of hyper-parameter tuning
· Don’t fall down the rabbit hole trying for the 10%
Course summary
Deep learning.ai and Fast.ai explain everything so clearly that you feel confident enough to take up the code and algorithms and start using them. The best way to learn, practice and reinforce my knowledge came from alternating between the two courses and data science competitions in Kaggle.
What are the recommended cloud platforms for running deep learning?
At some point, your laptop will not be fast enough to run deep learning models. Here are two options to mitigate that.
· First, you can reduce the complexity of your models, reduce the training data sets or fine-tune parameters and hyper-parameters.
· Second, get a Graphics processor. The good news is you don’t need to purchase a GPU, enterprise-grade GPUs are available to everyone in the cloud but be careful about the costs!
Google CoLab: This is a free platform that gives access to Google’s compute and GPU and Python libraries Keras, TensorFlow, PyTorch, and OpenCV.
Kaggle: Has a free coding environment, and you get 30hours a month GPU time. For beginners, this is enough, in my opinion.
Azure Studio: This eliminates the complexity of data engineering and Python coding, giving amateur data analysts and data scientists a more natural way to build models. Azure will charge based on your capacity so to keep costs down, make sure you shut down the instances.
AWS Sagemaker — Is an all in one AI platform. You can use AWS models, build your own or use AWS will host your models. Like Azure, you also get enterprise-grade graphics processing power, and there is a cost depending on the type of processing unit and memory. To keep costs down, make sure you shut down your instance when you finish. I created a simple AWS Lambda job to shutdown my notebook at specific intervals.
Kaggle and AWS Sagemaker were the best fit for my education workflows and experience. I developed the code on a laptop, ran with a small training/test set, fixed bugs, tuned and then uploaded to either platform for larger training/test sets and further tuning.
For more on AWS and Passing the AWS Machine Learning Specialty check out this blog post.
Design thinking is a human-centric approach to innovation and growth. It is based on understanding customers' needs, creative idea generation, and rapid prototyping
How can design thinking be applied to deep learning?
Artificial intelligence has the potential to improve our lives substantially. Still, it is essential to remember that AI is designed and created by humans for humans. Design thinking is a human-centric approach to innovation and growth. It is based on understanding customers' needs, creative idea generation, and rapid prototyping. The design thinking process balances desirability, feasibility, and viability to ensure that deep learning solutions are desirable for customers, technically feasible and can become a viable, sustainable business model. This allows organisations to make decisions based on customers' needs rather than only historical data or risky bets.
It involves several steps, including framing a question, empathising with customers, ideating, prototyping, testing, training a model, and continuously evaluating and refining the solution. When applied to a deep learning pipeline, design thinking ensures that the focus should be on using data science and deep learning to enhance human capabilities and experiences, guided by human centred design principles and the importance of privacy and ethics.
Deep Dive into Design Thinking's Integration with Machine Learning
Discover how design thinking can enhance the machine learning pipeline in this blog post Integrating Design Thinking into Machine Learning for Innovative Solutions. Learn about problem framing, empathy, ideation, prototyping, testing, iteration, implementation, monitoring, explainability, trust, and ethics. This resource offers valuable insights for developing human-centric machine learning models.
Deep Learning and the Evolution of Large Language Models (LLMs)
Large Language Models (LLMs), like ChatGPT, are a convergence of cutting-edge advancements in machine learning. When we dive into deep learning, a subset of machine learning, we find that it's primarily built around sophisticated architectures known as deep learning neural networks. These networks mimic the structure and functionality of the human brain, allowing for the extraction and understanding of intricate patterns from vast datasets. LLMs, in particular, employ transformer architectures from deep learning to handle and generate text. This capability, combined with generative AI techniques, empowers LLMs to produce coherent, contextually relevant content, making them exemplary tools in natural language processing and beyond.
Why is AI governance and ethics important in the context of deep learning?
The courses here will give you a deep technical understanding of how machine learning and deep learning work. As you start working on machine learning and deep learning in the real world, strong governance and ethical oversight cannot be added on later.
There are a burgeoning number of tools and information that help organisations understand and implement robust governance and ethical oversight. All of these sites have links to additional content.
The toolkit has in-depth information on governance, ethics, risk, cybersecurity and more. The toolkit goes into detail on board-level responsibilities for the adoption of AI.
The Principled Artificial Intelligence has eight key themes. Privacy, accountability, safety and security, transparency and explainability, fairness and non-discrimination, human control, professional responsibility and promotion of human values.
Deep Mind Safety and Ethics
The team addresses the complex ethical and social exploring the real-world impacts of AI.
“With the right focus on ethical standards and safety, we have better chances of finding AI’s potential benefits. By researching the ethical and social questions involving AI, we ensure these topics remain at the heart of everything we do.” (deep mind).
Ready to embark on a deep learning journey and unlock the potential of AI for your business?
Visionary Leaders Choose David Kolb Consultancy. Dive into a world where business strategy meets innovative problem-solving skills. Harness the creativity needed for today's artificial intelligence in business era. Your journey to a human-centred business transformation starts with a 1:1 session.
Looking for more insightful content?
Check out these related blog posts on business innovation, design thinking, AI and technology written by David Kolb.
From Innovation to Advantage: Harnessing the Power of Generative AI
Implementing Problem Solving: Strategic Approaches and Techniques for Organisational Growth
Merging Human Centered Design with Business Strategy (Video)
Enhancing the Machine Learning Pipeline with Design Thinking
Ethical Considerations for Boards in the Age of AI: Merging Technology and Values
Demystifying Artificial Intelligence for Boards: Uncovering the Power and Practicality
Navigating AI Adoption: The Essential Perspectives for Board Members
Or check out our weekly concise and valuable quick tips for visionary leaders and entrepreneurs.