General assignment information#
Getting started#
To edit/execute a Homework/lecture notebook:
Open the page for the Homework/Lecture on this site.
For example: Homework 0
Click the launch button (🚀) at the top.
JupyterHub may take a few minutes to start up.
You should now see the notebook in JupyterHub. That is now your own copy; make edits in there directly.
Tips#
Read the instructions carefully. Like word problems from math class, they are very specific in what they are asking for.
Ask for help.
The assignments are meant to be challenging, not impossible.
Try and work through problems on your own to start. If you are stuck for more than a half hour, step away. If you still can’t figure it out, ask for help.
Coding in general#
Make variable names descriptive.
Do as I say, not as I do!
Only do one thing per line.
Makes troubleshooting easier
Spot check your results. If you are transforming data from a previous Step, compare the results, do a handful of the calculations manually, etc. to ensure that the results are correct.
Don’t repeat yourself (DRY). If you find yourself copying and pasting code within your file(s), there’s probably a better way to do it.
Avoid hard-coding values. Don’t rely on things like row numbers or column order being stable, in case the dataset were to be updated.
Jupyter / JupyterHub#
All lecture slides and homework templates can be found under
python-public-policy/
. The contents of this directory will be automatically updated from the GitHub repository, but should keep any changes you make.FYI that this uses nbgitpuller under the hood.
Access JupyterHub via the links on this site rather than bookmarking JupyterHub, as that will pull down the latest changes.
Keep each cell to only a few lines of code.
Allows/encourages you to output intermediate results, ensuring that everything is working as expected.
You are always welcome to add cells. You probably don’t want more than a few lines of code in each. This makes the spot checking of intermediate results easier.
Make notebooks idempotent
Makes your work reproducible.
Use
Restart and run all
(⏩ button in toolbar).
Generative AI#
See also: course generative AI policy.
We will be using the AI magic.
Documentation#
Note that we only have the gpt-4.0-32k
model enabled.
Usage#
One-time setup: Visit projects.rit.nyu.edu and accept the Terms of Use.
This site is only accessible from the NYU network.
If you’re off campus, you can connect via the VPN.
Open JupyterHub.
Load the models by running the following in a
Code
cell:%load_ext rit_jupyter_ai_magics
In another cell, run:
%%ai gpt-4.0-32k how are you?
Replace
how are you?
with your prompt. You can repeat that cell as many times as you need.
Submission#
Ensure all the outputs are visible and the notebook is cleaned up.
This is a good time to run the notebook end-to-end with
Restart and run all
(⏩).
Export the notebook as a PDF. From the Jupyter interface, go to:
File
Save and Export Notebook As…
PDF
You may need to allow popups
Glance through the PDF to ensure everything is showing up as you intend.
In particular, check your visualizations.
What you see is what the instructors will see.
If one of the Homeworks: Upload the PDF to the Brightspace Assignment.
If the Final Project:
In Brightspace, go to
Content
, thenFinal Project
. You should see the TurnItIn/PeerMark dashboard.Follow these instructions to upload the PDF.
When you’re ready to have it formally re-graded, please resubmit through the same Assignment in Brightspace.
Note: In-class exercises will not be graded.
Common issues#
Error: Command '['git', 'diff', '..origin/nyu', '--name-status']' returned non-zero exit status 128
when trying to launch JupyterHub: Something got corrupted in your copy of the files that come throughnbgitpuller
. Easiest thing is to move the existing ones and restart with a fresh copy.There should be a
python-public-policy
folder. Rename it, something likepython-public-policy-old
.
If you’ve edited any of the template notebooks, you can find them in that
-old
folder.
Repeat the getting started steps.
When using
choropleth_map()
, nothing appears on the map: Make sure:Your
locations
corresponds to the DataFrame column name andfeatureidkey
is set toproperties.<property name>
matching the GeoJSONThe column and the GeoJSON properties have values that match
SettingWithCopyWarning
: How to fixinput()
stuck: Jupyter can be a bit buggy when dealing with interactive input. If it seems to get stuck or you aren’t seeing a prompt when you’d expect one, try clicking theKernel
menu thenRestart Kernel
.The values are out of order along the axis of a Plotly chart: Make sure that:
The values are integers/floats/timestamps, not strings, where applicable.
AttributeError: partially initialized module 'orjson' has no attribute 'OPT_NON_STR_KEYS'
: Add the following cell before your other Plotly code, thenRestart and run all
(⏩).import plotly.io plotly.io.json.config.default_engine = 'json'
Disk full#
If you get an error of Disk is full
/ No space left on device
/ Out of diskspace
: You’ve used all the available disk space. If you do fill it up, your server may not be able to start again (spawn failed
). You’ll need to delete one or more large files that you don’t need anymore:
If you server is started already (you’re seeing notebooks), click
Control Panel
->Stop My Server
.Go to start your server again.
Select
Troubleshooting Only - Clear Disk
.Look at the
File size
Jupyter shows in the file browser.Delete one or more large files.
If you’re still using those datasets, make them smaller.
Error loading notebook
#
This error can happen if you tried to output a lot of data in tables/charts. Steps to resolve:
Open the JupyterHub) file browser
Click
New
, thenTerminal
Run the following, changing the path at the end to match whatever notebook needs to be repaired:
jupyter nbconvert --to notebook --clear-output ~/python-public-policy/hw_<NUMBER>.ipynb
If you’re confused by these instrucions, download the notebook file and email to the instructor.
kernel and memory issues#
The kernel is the place where Python is installed and the code is actually executing, in the cloud somewhere.
Make sure
Python [conda env:python-public-policy]
is selected as the kernel.Shows in the top right of the notebook interface
To change:
Open the
Kernel
menuClick
Change kernel
Click
Python [conda env:python-public-policy]
If your kernel is repeatedly crashing, you’re probably running out of memory.
Make sure you aren’t loading data sets you don’t need.
If loading a new dataset, make it smaller
Close kernels you aren’t using from the Running page.
PDF export#
Jupyter notebook export to PDF is fragile, especially with interactive charts through Plotly.
If you get a 500 error, it could be happening for a handful of reasons. Scroll to the bottom and read the message.
If it mentions
Undefined control sequence
and\pandocbounded
, it’s an issue with the exporter. Jump to the Alternatives.If it mentions a package being missing, ask on Ed.
Otherwise, it may be crashing due to not being handle the complexity of the render. Try reducing your output (in the Jupyter sense) to smaller subsets. This can include:
Not displaying so many rows/values
Reducing the number of points that are plotted
Alternatives#
If you are unable to get the direct-to-PDF export working, try:
Going through HTML:
Open the HTML file in your browser.
File
->Print…
Save as PDF
If any of the visualizations are cut off:
Cancel the Print to PDF.
Viewing the HTML file, make the browser window narrower.
This forces the visualizations to re-draw.
Try from
File
->Print…
again.
An online converter, such as Ploomer