If you are someone who wants to use deep learning to predict the future, this course is for you!

Deep learning using Neural Networks is a powerful technique that can solve many problems involving predicting future events. This course aims to take you beyond the hype to give you the confidence to apply neural networks to real-world predictive problems. This 5-day course aims to teach you:

**Neural Network (NN) techniques for timeseries forecasting.**Using the Caffe deep learning framework, we will teach you how to create NN models for timeseries forecasting, which can be applied to any area that requires you to predict future events given time-varying data sources.**Understand how NNs work.**To use them effectively, you have to really understand how NNs work. What they can and can't do. We will take a bottom-up approach to teaching you how to translate the complex math into practical knowledge you can use to design and train NNs. This is reinforced with quizzes and 10 hands-on lab sessions.**Best practices for NN based forecasting.**How to determine optimum network configurations? How to train complex networks? How to evaluate and track performance? When to apply a technique and when not to? We teach you how to develop an intuition of what will work and what won't so you can reason for yourself.

In short, this course will help you design effective neural networks for solving real-world forecasting problems.

Anyone who works with time-series data or needs to forecast future events should find this course very helpful. Some possible fields of application:

**Engineering:**predictive maintenance, failure prediction, load forecasting, system modeling & optimization.**Planning & Operations:**supply chain forecasting, logistics planning, process optimization.**Finance & Economics:**price forecasting, banking, economic forecasts.**Environmental & Geophysics:**Forecasting floods, earthquakes.**Agriculture:**Forecasting crop yield, crop failure.**Medicine & Biology:**Disease outbreaks (we will cover a specific instance of a vector borne disease, Dengue).

The lectures and labs will be backed by a real-world problem: predicting the outbreak of dengue [2] in Singapore using actual environmental data.

Each Day is split into two sessions, each comprising a Lecture followed by a Lab. (ie, 2 lectures and 2 labs a day). You should attend all sessions as each session builds on previous ones. The Labs are an integral part of the learning experience, and you should complete every Lab. In addition to Lectures & Labs, we also include Quizzes throughout the Lectures to help you challenge your understanding of the concepts. Please complete these at home because we will discuss them on the next Day.

Also, each Day sometimes ends with a take home mini-project that you are encouraged to attempt before the next Day's sessions. We've designed these mini-projects to greatly enhance your learning.

Lastly, we have an online forum so you can continue to ask questions well after the workshop.

- You should preferably know a programming language (eg, Python). While this is not a course in programming, some programming can be helpful for preliminary data cleaning and data analysis. Python is an excellent choice in this regard and easy to learn. We expect you have rudimentary programming ability in this course.
- You should be comfortable with basic multivariate Calculus (specifically differentiation over multiple variables). This background is helpful, but not strictly necessary, in appreciating the mathematical underpinnings of NNs.
- You need a decent laptop. All lab sessions will be done on your laptop using Virtual Box[3]. Your laptop needs to be: 64-bit, have at least 10GB of free disk space and 4GB of RAM. Bring your laptop fully charged with your charger to all workshop sessions. Most modern laptops (1 - 3 years old) should satisfy this criteria. Ipads, Android tablets and Netbooks can't be used to run software for this course.

An early version of this course was conducted by Terra Weather in Dec 2017 at the National University of Singapore as part of IoT Datathon 2.0 during which 110 students mainly from non-computing backgrounds (biology, physics, chemistry, statistics, mathematics, business and mechanical/electrical engineering) learnt deep learning techniques for timeseries forecasting to solve real-world problems.

Some feedback from participants:

"The lectures are very good in explaining deep learning concepts. The lecturer provides thorough elaboration on building deep learning models, especially for time-series forecasting on rare events. Solving the problem set given during the workshop provides us realistic perspective on applying deep learning techniques in real-world scenario." - Valluru Chandana, Research Associate at Saw Swee Hock School of Public Health

The deep learning concepts taught during the workshop was excellent, not too overwhelming but more than adequate to solve the problem sets given by the company. The fact that Terra Weather introduced real-world problem set during the workshop allowed me to apply the concepts right away and learn from the experience, which is something that online courses or other competitions do not offer." - Sean Ng Peng Nam, Year 4 Computer Science student at NUS School of Computing

"At first, I had no clue about neural network and artificial intelligence. The workshop was an eye-opening experience for me as the lecturer provided essential pieces of deep learning concepts which is required to build AI models and solve real-world problems. It gives me ability and confidence to pursue career related to deep learning in the future." - Muhammad Saifullah, Year 4 Physics student at NUS Faculty of Science

**Samuel Wang** holds a masters' degree in Physics from the National University of Singapore (NUS). He is a Data Scientist at Terra-AI.SG and will be the lead trainer for this AI Workshop. Samuel has contributed to the development of TW Caffe (see http://ai.terrawx.com), our open- source fork of Caffe specifically aimed at timeseries forecasting. He also works on Autocaffe, a productivity tool to simplify deep learning on Caffe.

**Arnold Doray** holds a degree in Physics and masters in Knowledge Engineering from NUS. He leads product development at Terra Weather and is the lead developer of Autocaffe and the Smojo Programming Language we use for data processing. Arnold is the alternative trainer for this AI Workshop. Arnold is an industry veteran with over 20 years' experience in implementing AI and automation in various organizations, including leading change in our parent company, Terra Weather. The result has been significant improvements in quality and customer satisfaction as well as extensive cost-savings through better use of manpower.

Contact Mr Christian Jonathan 6515 4775 or email info@terra-weather.com.

Venue: | Devan Nair Institute for Employment and Employability 80 Jurong East St 21, Singapore 609607. |

Time: | 5 days. 9:00 am-5:00 pm |

Fees: | $4000. |

Lecture 1: Introduction to Caffe & Linux

- Brief overview popular frameworks. GPU & CPU. CuDNN, MXNET; Keras/Theano/Tensorflow.
- Data exploration tools (Jupyter Notebook)
- What is Caffe. Why we use it. Why it is good to learn it.
- Terra Weather's additions to standard Caffe / Autocaffe, for automating Caffe.

**Lab 1:** Using VirtualBox. / Basic BASH commands. cd, ls, vim, ssh, scp, cp, mv, rm, chmod running shell scripts. Simple hello world in BASH / Getting Jupyter Notebook to work. / Transfer files into/out of Linux.

Lecture 2: Running Caffe

- Caffe overview of mechanics: Prototext file format, solver file, resources.
- Caffe's dimensions.
- Creating new input data using Python & HDF5.
- How to examine & visualize the output?
- Introduce autocaffe functions for scheduling work.
- Introduce autocaffe network generation using Smojo scripting.

**Lab 2:** Running toy example: Sine wave prediction / Double sine prediction.
**Homework:** Construct Median, Min, Max of weekly temperature & rainfall data into HDF5 format from raw datasets. This homework requires some Python programming background, which we will cover at the end of the session.

Lecture 3: Neural Networks Basics

- What they are (basic building blocks) / Why they work. / Interpretation of output as class, probability or value.
- Simple networks ( MLP ) with simple loss layer.
- Risk Minimization

Lab 3: Writing simple Smojo scripts.

Lecture 4: The Backpropagation Algorithm

- Backpropagation Algorithm / Mathematical Underpinnings
- Caffe's forward & backward passes.
- Training / Testing

**Lab 4:** Predicting the median temperature 1, 4, 8 weeks ahead.

Lecture 5: Training and Evaluating Neural Networks

- How good is good? Using benchmarks to evaluate performance.
- Need to collect statistics by re-running experiments.
- Mini-batches.
- Overfitting: What it is / Detecting / Early Stopping / Regularization / Dropouts.
- Using AutoCaffe for simple visualization with Latex templates.

**Lab 5:** Predicting the median temperature 1, 4, 8 weeks ahead ("time horizon") with persistence benchmark. Try predicting rainfall (optional)

Lecture 6: Timeseries Data

- What sets apart timeseries data?
- NNs are not black boxes / need for preprocessing & good loss functions.
- Basic Input Preprocessing: Input Normalization / Why it is important.
- Eliminating Dependent input variables / Why it is important.

**Lab 6:** Using custom layer extensions to Caffe.

Homework: The dengue dataset. Simple networks to predict dengue. Compare predictions with Persistence. Participants should try 1, 8 & 16 week predictions.

Lecture 7: The Loss Function

- Importance of Loss function / Why naive losses are not sufficient for timeseries predictions.
- Loss functions: Euclidean / Momentum / Force / How these work.
- Evaluating performance: Autocorrelation & Lag / Why lag itself is not sufficient.

**Lab 7:** Implement Input scaling & Momentum & Force losses for dengue predictor. Same time horizons.

Lecture 8: Deep Learning & Stacked Autoencoders.

- What is "deep" learning? / Wide vs Deep / feature extraction and why it is important.
- Challenges of training deep networks & solutions: Layer-wise training / Memory using gates (LSTMs, GRUs)
- Stacked Autoencoders (SAEs): what they are / Why they are valuable / How to train them / How to use them for predictions. / feature extraction / When not to use them.

**Lab 8:** Participants will construct & train a 3-level SAE for dengue input using Autocaffe's prefabs.
Homework: Retry the dengue prediction with a 5-layer SAE. Same time horizons. Read the paper on earthquake prediction.

Lecture 9: Data Transformations

- Avoid dependencies in inputs. NNs are not black boxes.
- Using Moments -- what are they / why they are useful for time series data / How to do this using Autocaffe.
- Thresholding
- Fourier transforms -- what it is / when is this useful?

**Lab 9:** Retry dengue prediction with moments and thresholding of inputs.

Lecture 10: Putting it All Together

- An efficient NN design and training workflow from start to finish - how to begin designing networks for real world problems?
- We will use the Dengue problem set to illustrate this workflow.

**Lab 10:** From prototype to production - making your predictive system trusted by end-users. We design and deploy a working, realtime web-accessible system to deliver and monitor your dengue predictions.

- http://caffe.berkeleyvision.org/ Caffe is a deep learning framework originally meant for machine vision. The emphasis is on speed. Terra Weather have adapted Caffe for timeseries forecasting.
- Dengue is a vector bourne disease, carried by the Aedes aegypti mosquito. This disease is strongly affected by the weather.
- https://www.virtualbox.org/wiki/Downloads

Terra-AI.SG is a division of Terra Weather, a technology company specializing in developing cutting-edge prediction technology and Artificial Intelligence based solutions. Our weather AI processes data from multiple sources such as satellites and sensors and converts them into highly accurate weather predictions. We enable multi-national Oil and Gas, Marine Transportation and Marine Consultancy companies work safely and successfully worldwide.