IFT 3710/6759 - Projects

The chief learning objective of this course is to prepare students for tackling real-world machine learning projects. Therefore, all students will participate in teams in the development of one machine learning project. This page describes the general guidelines about this part of the course.

General information

The projects of the course are aimed to resemble as much as possible what real-world machine learning projects look like in either industry or research. Projects comprise the following stages:

  1. Literature review
  2. Planning
  3. Development
  4. Analysis of results
  5. Written report
  6. Oral presentation

Timeline

The first four weeks of the course are devoted to introducing the course, reviewing the core aspects of machine learning and deep learning, and important tools for carrying out machine learning projects, such as Linux, git and PyTorch. After that, you are expected to work on the assigned project from week 5 to week 14. As a final deliverable of the project, you will prepare a short (~8–10 pages) technical report and an oral presentation to be given in front of the teachers and the other students. During the development of the project, there will be intermediate milestones and deliverables to facilitate smooth progress. The specific timeline is the following:

  • Week 4, February 1st 2022–February 4th 2022: review of proposed projects, expression of project preferences and proposal of own projects.
  • Week 5, February 7th 2022–February 11th 2022: project assignments, teams formation, first meeting with instructors.
    • By the end of the week, all teams should be formed and registered, every student should be in a team.
    • Deliverable 1 (February 11th 2022): team registration: working title and members.
  • Week 6, February 14th 2022–February 18th 2022: project planning, literature review.
    • Deliverable 2 (February 18th 2022): project plan: one-pager containing a brief description of the project, foreseen steps, goals and tentative distribution of work among team members.
  • Week 7, February 21st 2022–February 25th 2022: literature review, first steps.
    • Deliverable 3 (February 25th 2022): literature review: one-pager containing a review of relevant, related literature.
  • Week 8, February 28th 2022–March 4th 2022: break (période d’activités libres)
  • Weeks 9–10, March 7th 2022–March 18th 2022: project development (phase 1)
    • Deliverable 4 (March 18th 2022): methods and preliminary results: one-pager containing a description of the methods developed and planned, relevant changes to the original plan, and preliminary results and/or challenges.
  • Weeks 11–12, March 21st 2022–April 1st 2022: project development (phase 2)
    • Deliverable 5 (April 1st 2022): results and conclusions: preliminary report of results and conclusions (two pages).
  • Weeks 13–15, April 4th 2022–April 20th 2022: project development (phase 3), preparation of presentations and final report.
    • Deliverable 6 (April 20th and April 22nd 2022): presentations in class
  • Week 16, April 25th 2022–April 29th 2022: preparation of final reports.
    • Deliverable 7 (April 29th 2022): final report

Project assignment

The projects will be developed in teams of 3–5 students. A set of projects will be proposed by the instructors, and you are also welcome to propose your own projects (see below. At some point during the introductory weeks, you will be asked to form teams and to provide your preferences about the suggested projects or proposed your own, in order to distribute the available projects among all teams the best possible way. Some projects may be easier than others, which will be taken into account in the evaluation. If a team (or a student) concludes one project before the end of the course, they will be able to work on a new project for extra points.

Project proposal

In order to foster creativity and allow you to work on projects of your interest, you are welcome to propose your own projects. Nonetheless, the project proposal must be accepted by the instructors and the decision will be based on the following criteria:

  • The project must involve the use of advanced machine learning methods.
  • Worked developed prior to this class will not be accepted.
  • It must be feasible in terms of computational resources and time constraints.
  • The data must be publicly available.
  • The project should not raise major ethical considerations.
  • Projects that tackle real-world problems with a potentially positive impact will be favoured.

Evaluation

Students will be evaluated entirely according to their work on projects and the final grade will be based on the following criteria, all with equivalent weight:

  • Difficulty of the project
  • Quality and performance of the developed algorithms: suitability of the chosen methods, technical rigour, results, etc.
  • Written report: completeness, clarity, technical soundness, analysis, etc.
  • Oral presentation: effectiveness, clarity of the presentation, etc.
  • Code: clarity, documentation, modularity, extendability, etc.

The evaluation will be at the project level, that is we will consider that the results, the report, the presentation, and the code are the output of the whole team. Detailed instructions regarding the presentations, reports and code are available here. Nonetheless, the grades may be adjusted individually if necessary in the case of participation imbalance (see below).

Important notes:

  • The evaluation criteria will be slightly relaxed for undergraduate students (IFT 3710).
  • The grade will be binary (pass or fail), not in a letter scale.
  • The teams should not mix graduate (IFT 6759) and undergraduate students (IFT 3710).

Teamwork

Teamwork is an essential feature of machine learning projects, and engineering projects in general, both in research and industry. Therefore, it is a requirement of this class to work on the projects in groups of at least three students. We recommend to form teams of up to five people, but slightly larger teams may be allowed depending on the difficulty and characteristics of the project.

Working in teams does not necessarily mean that every teammate contributes equally to every part of the project. It is up to each team to design their most suitable distribution of the workload. However, we do expect every student to engage in all stages of the project (literature review, design, coding, analysis, writing, presentation, etc.).

In order to encourage effective teams, mitigate imbalances and ensure a degree of fairness in the evaluation, we will adopt several mechanisms, which include:

  • Students are free to organise themselves and propose teams to work on specific projects.
  • Otherwise, teams will be formed according to the preferences provided by the students.
  • During the development of the projects, we will carry out several team-wise meetings with one of the instructors, to not only assess the progress in the project, but also the functioning of the team.
  • At the end of the project, each team member will have the opportunity to fill a questionnaire about the functioning of the team and the contributions to each stage of the project.

Communication

During the project development phase, we will facilitate a weekly session (online, or in person) of each team with one of the instructor (main instructor or a teaching assistant). The goal of these meetings is to monitor the progress of the project, the smooth functioning of the team and to give you the opportunity to ask questions and get feedback. For daily communication, you are welcome to participate in a Discord server set up for this class. Please, do send me an email (alejandro.hernandez.garcia@umontreal.ca) if you do not have access to the Discord group. You are also welcome to set up a channel for your team within this Discord server, or use a different means of communication of your preference.

Finally, do not hesitate to contact the instructors if problems arise. We are here to help!

Expectations and potential publication

The goal of this course is learning. That is, you will be expected to work on a problem where machine learning can be applied, perform a literature review, analyse your results, write up a report and present it to the class. Doing all this will certainly guarantee that at the end of the process you will be better prepared for tackling other real-world machine learning projects. Doing all this in a sound way will also certainly guarantee a high grade. We definitely do not expect you to produce a research paper ready for publication. Nonetheless, if you are interested in pursuing a research career or in publishing your work for any other reason, note that this kind of project does resemble to a great extent a research project that could lead to a publication if the outcome can be of interest to a research community. Note that this would probably require extra work after the course is over. If publishing your work is something you would like to consider, feel free to discuss this with the instructors for feedback at any stage of the project.

Proposed projects

As explained above, you may propose your own project or work on one of the projects proposed by the instructors. Below is the list of projects proposed by the instructors and additional details can be found in the page of each project (clicking on the titles):

Final projects

Animal detection with Faster R-CNN: transfer learning and domain adaptation. Abdiel Fernandez, Rose Guay Hottin, Kevin Lessard, Santino Nanini PDF

Analysis of image augmentation methods on different types of learning problems. Alekhya Dronavalli, Krishna Maneesha Dendukuri, Prishruit Punia, Rakshit Shetty, Timothy Lee, Yassir Mamouni PDF

Photovoltaic power and solar radiation forecasting. Kodjovi Adabra, Marcos A. A. Souto Jr., Niranjan Niranjan, Rui Ze Ma PDF

Conversational question answering system. Charmi Chokshi, Hena Ghonia, Sandeep Kumar, Vamsikrishna Chemudupati PDF

Inductive biases in vision classifiers. Abhay Puri, Axel Bogos, Pulkit Madan, Jizhou Wang PDF

Analyzing the performance of transfer learning with different EEG datasets. Ronnie Liu, Anas Bourhim, Hichem Sahraoui, Van Nam Vu PDF

WILDS - Distribution shifts - iwildcam. Vijaya Lakshmi Kuruba, Saif Kurdi-Teylouni, Chaitanya Kothapalli PDF

ANimal Deep learning Identification -ANDI. Jean-François Baril, Jonathan Beaulieu Emond, Yassine Kassis, Selim Gilon PDF Slides online

FMoW-wilds: Land use classification across different regions and years. William Callaghan, Jérémy Parent, Sorin Muchi, Nathan Alix-Vignola, Mathieu Lamarche PDF