General assignment information

• Due dates

• Scoring

Getting started

To edit/execute a Homework/lecture notebook:

1. Open the page for the Homework/Lecture on this site.

   • For example: Homework 0

2. Click the launch button (🚀) at the top.

3. You should now see the notebook in Google Colab.

4. Make sure you’re using your Columbia Google account. How to switch.

5. Click Copy to Drive. That is now your own copy; make edits in there directly.

Tips

• Read the instructions carefully. Like word problems from math class, they are very specific in what they are asking for.

• Spot check your results. If you are transforming data from a previous Step, compare the results, do a handful of the calculations manually, etc. to ensure that the results are correct.

• You are always welcome to add cells. You probably don’t want more than a few lines of code in each. This makes the spot checking of intermediate results easier.

• Don’t repeat yourself (DRY). If you find yourself copying and pasting code within a notebook, there’s probably a better way to do it.

• Avoid hard-coding values. Don’t rely on things like row numbers or column order being stable, in case the dataset were to be updated.

• Ask for help.

  • The assignments are meant to be challenging, not impossible.

  • Try and work through problems on your own to start. If you are stuck for more than a half hour, step away. If you still can’t figure it out, ask for help.

    • Ed

    • Office hours

Storing data

To keep data between sessions, you’ll probably want to store it in Google Drive.

1. Upload the file(s) somewhere in Drive.

2. In the Colab sidebar, click the Files icon.

3. Click the Mount Drive icon.

   • Think of this as attaching your Drive to your Colab instance, as if you were plugging in a USB flash drive.

4. Navigate to the file.

5. Next to the filename, click the three dots.

6. Click Copy path.

   • The value should be something like /content/drive/My Drive/....

7. Use this path with read_csv().

Reducing data size

You can make data smaller before uploading by filtering it through:

• The data portal, if it supports it

  • This makes the download faster, including only the data you need.

  • Instructions for Socrata-based portals

• In a spreadsheet program

Submission

1. Ensure all the outputs are visible and the notebook is cleaned up.

   • This is a good time to run the notebook end-to-end with Restart and run all.

   • See general scoring criteria.

2. Confirm that the notebook is shared with the instructor and Reader with Commenter permissions. If it isn’t, share the parent folder and re-confirm.

3. Copy the URL of your notebook.

   • The URL should be of the format https://colab.research.google.com/drive/<long identifier>. If it’s https://colab.research.google.com/github/..., click Copy to Drive.

4. Paste your notebook URL in the CourseWorks Assignment.

Engaging with comments left in Google Colab is more than welcome. After the late submission deadline for a given Homework passes, the solution will be shared.

Note: In-class exercises will not be graded.

Common issues

• Mounting Google Drive is slow or fails: See the Google Colab help page.

• Can’t load a file from Drive with requests.get(): Use open().

  • How to read a JSON file

• When using choropleth_map(), nothing appears on the map: Make sure:

  • Your locations corresponds to the DataFrame column name and featureidkey is set to properties.<property name> matching the GeoJSON

    • See how we found the property name to use

  • The column and the GeoJSON properties have values that match

• SettingWithCopyWarning: How to fix

• input() stuck: Jupyter can be a bit buggy when dealing with interactive input. If it seems to get stuck or you aren’t seeing a prompt when you’d expect one, try clicking the Runtime menu then Restart Kernel.

• The values are out of order along the axis of a Plotly chart: Make sure that:

  • The values are integers/floats/timestamps, not strings, where applicable.

  • Line charts: The column used for the X axis is sorted.

• AttributeError: partially initialized module 'orjson' has no attribute 'OPT_NON_STR_KEYS': Add the following cell before your other Plotly code, then Restart and run all.

  import plotly.io
  
  plotly.io.json.config.default_engine = 'json'
  

runtime and memory issues

The runtime is the place where Python is installed and the code is actually executing, in the cloud somewhere.

• If your runtime is repeatedly crashing, you’re probably running out of memory.

  • Make sure you aren’t loading data sets you don’t need.

  • If loading a new dataset, make it smaller