####################################### # LING 300 # Introduction to Programming # and Text Processing for Linguists # Winter 2021 ####################################### Welcome to week 1! This file is your assignment for the week. We'll be settling in to using the terminal, logging in to Quest, and so on. The last part of the assignment will involve using what you learned to submit an edited version of this file on Quest and set the appropriate permissions. Notation: - I'll use >>> to indicate where to put your answers. - When I mention a command inline you can run, I'll surround it with backticks like so: `cd`. Important Unix notes: - Most (but not all) Unix commands and programs can be quit out of by typing Ctrl-c. Use `q` to quit `man` or `less`. - You can exit your terminal with the command `exit`, and use this as well to leave an SSH session and return to your local shell. Once you've completed the assignment, come back up here (in your command-line text editor) and fill these questions out: Name: >>> Approximately how many hours did this take to complete (including time working on it in class)? >>> Any comments or questions? >>> ###################### # Core Exercises (1-8) ###################### #### 1. Log in to Quest Quest is Northwestern's "high-performance computing environment." What that means is basically it's a group of computers sitting somewhere on the Northwestern campus that we'll access and use remotely, wherever you are. We'll do most of the work for this class on Quest. Your computer will provide a terminal that will log in to those remote computers using a tool called SSH ("secure shell"). You'll then be able to manipulate files and run programs on that remote system. If you ever run into issues, you can always close your terminal or log out of your session and try again to restart. Here's the Quest documentation on logging in: https://kb.northwestern.edu/internal/page.php?id=70705 But I'll explain in a bit more detail. First things first, open a terminal. - If you're running Mac OSX, the Terminal app will be in the Utilities folder in Applications (or you can search "Terminal" in Spotlight). - If you're running Linux, it differs by distribution, but a terminal of some sort should be among the default applications on your system. On Ubuntu, for instance, you can open one with the keyboard shortcut Ctrl+Alt+T. - If you're running Windows, you have two choices. One is to download a remote terminal emulator like PuTTy (https://www.chiark.greenend.org.uk/~sgtatham/putty/) or MobaXTerm (https://mobaxterm.mobatek.net/, which I prefer over PuTTy). If you have Windows 10 the other option is to install the Windows Subsystem for Linux (https://docs.microsoft.com/en-us/windows/wsl/install-win10, I recommend installing Ubuntu 18.04 LTS). This latter option is better if you want to play around with anything on your local machine rather than Quest, since it can run Linux command line tools on Windows natively - PuTTy and MobaXTerm can only connect to remote servers. Whichever OS you're running, once you have a terminal open, simply run: ssh netid@quest.it.northwestern.edu Be sure to replace netid with your netid (eg abc1234@quest.it.northwestern.edu). You'll be prompted to type in your netid password - type it in (generally the characters will be invisible, don't worry they're there) and press enter. Depending on your terminal you may have to type 'yes' to accept connecting to a new node (computer) your machine hasn't seen before. Note: The Quest documentation recommends adding the '-X' flag, which allows the terminal to pass graphical user interfaces (GUIs) through from the remote server. I recommend not doing this unless you have a good reason to, because they are likely to be very slow (especially if you're not on campus). If you're using PuTTy or MobaXTerm on Windows, this will be slightly different - you'll not directly run the SSH command, but rather input the settings for a new SSH session, which look like this: Hostname : quest.it.northwestern.edu Username : your Northwestern NetID Password : your Northwestern NetID password and then click connect. #### 2. Set up your homework directory Now you're in Quest! You'll see a bunch of introductory text at the top. Once you've logged in, you'll be in your home directory (~). If you list the directory (ls) it will likely be empty and thus print a blank line. As a class we have an "allocation" on Quest - a directory and allotment of space from the university - located at the following path: /projects/e31086/ Navigate to that directory (`cd`), and list its contents (`ls`). You should see a 'data' directory and a 'users' directory: `cd` into 'users'. Use `mkdir` to create a directory named after your netid (so for instance mine is rfj5679), and `cd` into it. Use `mkdir` again to create a directory named 'assignment1', and `cd` into that. Now you're in the directory where you'll put your assignments for me to check out. For reference, you can see the full path where you are at any time by running the `pwd` command, which stands for "print working directory" - if you run it now, you should see "/projects/e31086/users/[your_netid]/assignment1". #### 3. Obtain and look at this file Now you'll get this file into your homework directory so we can start working with it. `wget` is a command for obtaining files from the internet. Copy this url from your web browser. While still in your assignment directory, use the `wget` command with the url of this file as an argument to download it. Note: Copy-pasting the url into your terminal might be slightly different depending on which OS and terminal program you're running, so that's one thing to figure out because it will be useful over and over. Now if you run `ls`, you should see "assignment1.txt" in the directory. Run `cat assignment1.txt`; it will print the entire contents of the assignment to the terminal. `cat` stands for concatenate, because you can list multiple files as arguments and they'll all print concatenated one after the other. Run `less assignment1.txt`; this opens a program called `less` which is useful for viewing text files. You can navigate with your arrow keys and pgup/pgdown, and search by typing '/' then your query. Typing '/'+enter after a search goes to the next found instance of the search. An *extremely* useful command for understanding files you're working with is `wc` (for "word count"). Look it up with `man`. Do these problems before editing this file at all, and come back and fill in the answer after you've figured out your text editor in the next section. a. According to `wc`, how many words are in this assignment? >>> b. How many lines are in this assignment? >>> c. How many characters long is the longest line in the assignment? >>> #### 4. Working with a text editor Human-readable text is a "universal interface" for working with computers on the command line, and programming is basically a process of writing down what you want the computer to do. So text editors are a well-developed sort of program with many options and a long history, and not a small amount of controversy as far as what people prefer. Here are the main possible text editor programs to choose from: `nano` The simplest option to get started. It opens a text buffer - a screen to write text on - and displays the available commands at the bottom of the screen. Note that ^ refers to the Ctrl key, so ^X will exit the program. If you plan to use nano regularly, I strongly suggest you use it to create a file called ".nanorc" in your home directory and add these lines: set tabsize 4 set softwrap include "/usr/share/nano/python.nanorc" include "/usr/share/nano/sh.nanorc" Files starting with a period are called 'dotfiles', and they're a type of hidden file - for instance, they won't show up when you run `ls` unless you add the '-a' flag (for 'show all'). Often dotfiles are used to set various sorts of configurations on the command line. In this case nano will read this .nanorc file every time you open nano and run the commands in those lines, which will make it so your tab size is 4 spaces (instead of the default 8) and the appropriate syntax for command line scripts and python code is highlighted (will be very useful in later weeks). `emacs` Easy to learn, a lifetime to master. Very extensible with a billion features, can even be a calendar, web browser, todo list, etc, and some people organize their whole lives in it. There is a useful cheatsheet of commands here: https://www.gnu.org/software/emacs/refcards/pdf/refcard.pdf The most important ones to know are: Ctrl-x then Ctrl-s (save/write file) Ctrl-x then Ctrl-c (quit) `vim` Harder to learn, also a lifetime to master. It's a "modular" editor meaning it switches between modes that do different things like insert, replace, and select. A good quick intro lesson is here: https://www.youtube.com/watch?v=ggSyF1SVFr4 You can't actually type in the normal mode, you can only navigate and manipulate text (like deleting entire lines). You press 'i' to switch to insert mode which allows you to more or less type normally, then pressing Esc returns to "normal" mode. Commands are generally prefaced with a colon. The most important commands in normal mode are: :w (save/write the file) :wq (save/write the file and quit) :q (quit) :q! (quit discarding changes) Ultimately if you keep programming it will pay off to learn emacs or vim. Nano will be enough for this class, but I actually recommend emacs since it's a happy medium - much easier to learn than vim, but with some nice features built right in like automatic code highlighting and spacing. And with emacs as long as you know how to save and how to quit, you can just open it and type normally. #### 5. Introductions! Now that we've got some basics down of how to move and shake in the terminal, and how to edit text files, I just want to know more about you! Use your text editor to fill in answers to the questions below. a. What is your computational background? >>> b. What is your linguistics background? >>> c. Why are you interested in taking this class? What are your goals for this class? >>> d. Do you have any preliminary thoughts on what you might like to do for a final project? If you have existing research or projects that could be helped by these methods, I encourage double-dipping. If you don't know yet, that's okay too. >>> e. Is there anything else you want me to know about you or your situation coming into this class? This could include things like names or pronouns, learning style, questions about whether this class is a good fit, concerns about doing things remotely, or whatever else you'd like to share. >>> #### 6. Navigation, tab completion, and shortcuts At this point I recommend opening two terminal windows, one with this file open in your text editor so you can write answers, and the other available to do the exercises. Otherwise you'll have to keep closing and re-opening the assignment to do anything. Run the SSH command again to get into Quest on the new terminal - you'll notice you can be logged in simultaneously in both terminals, no prob! a. Go to the directory /projects/e31086. Type `cd `, and press the tab button on your keyboard twice. What happens? Is it like the output from any other program you've used? >>> b. What if you press tab twice after `cd` with no following space? What do you think this output means, and why is it different without the space? >>> c. Run `cd` by itself, with no directory specified after it. What happens? >>> d. Command History: You can navigate through your history of commands with the up and down arrows. You can also search through your history by typing Ctrl-r: this opens a prompt to type in some search material, and while searching pressing Ctrl-r again cycles through commands in the history that match. Get back to the course directory without typing out the directory name. How did you do it? >>> e. Try running `cd -` a few times. What happens? >>> f. Another sort of shortcut in Unix is the symbolic link, which makes a reference to another location in the filesystem. We do this with `ln -s`. First navigate to your home directory (~). Make a link to your /projects/e31086/users/[netid] directory by providing it as an argument to `ln -s`. What happened? (Check things out with `ls`) >>> #### 7. Permissions! Unix has a somewhat intimidating but ultimately very useful system for dealing with file permissions. You can get more information here (http://linuxcommand.org/lc3_lts0090.php), but the summary is that each file has 9 bits of permission settings to it: read, write, and execute permissions for each of the user, the group, and global (everyone else). You are the user, our class is your group, and we aren't concerned with anyone else. a. Go back to your home directory. Use your text editor to create a file there called ".bashrc" (or open it to modify if it already exists). `.bashrc` will be read by the shell upon startup to look for settings, so we're going to put some in. Insert (or modify to match if it exists) the following line: umask 002 b. Save the file and close it. Now log out of all your Quest terminal windows, and log back in. Doing this sets default file permissions so they're correct for any new files you create. `umask` is the command to set this, the first 0 means you (the user) can read/write/execute, the second 0 means group members can read/write/execute (so Thomas and I can grade your work by writing in it), and the last 2 means others can read but not write or delete your work. This last one is somewhat irrelevant since our project directory itself is locked to anyone outside of the class, so no need to worry about spies here. But, `umask 002` is a very common setting you might see elsewhere. c. Now in your assignment directory, make sure this current file has the right permissions. Do this by running `chmod 775 assignment1.txt`. `chmod` is the per-file equivalent of `umask` (which sets the default), and 775 is equivalent to umask's 002. The reason, which you don't really need to know, is that umask does subtraction from 777 (which == no restrictions, full permissions for everyone). So 777 - umask == chmod. Sort of. Anyway don't worry about it at the moment. #### 8. Navigation and file creation a. In your home directory on Quest, use `mkdir` repeatedly (navigating around if you need to) to make a nested series of directories that looks like this: ~/this/is/a/deep/directory b. Use `touch` to create a file called 'is_just_a_file' in the 'this' directory. c. `cd` all the way in to the 'directory' directory. `touch` a file called 'and_a_hiding_file'. d. Just as '~' is a special character representing your home directory, '..' is a symbol referring to the directory one level up in the hierarchy. You can chain these symbols to refer to directories far upward, e.g. '../../../../../' refers to the directory five levels up in the hierachy. Without leaving the deepest directory, use `touch` to create a file called 'another_file' in the 'is' directory. e. `cd` back to your home directory, and try to delete the 'this' directory with `rm`. What happens? >>> f. Since we're in your home folder, you should have the symbolic link to your ling300 user directory in here. Try to delete that with `rm`. What happens, and why? (Fix it if anything broke!) >>> g. Now using `rm` but without using `cd`, delete the 'is_just_a_file' file. h. I/O redirection - this was mentioned in the "Missing Semester" video lecture, but we'll talk about it much more next week. For now the thing to know is that `>` takes the output of some command to the left of it, and saves it in the filename given to the right of it (creating the file if it doesn't exist). Using `echo` and `>`, create a file called 'is_yet_another_file' in the 'this' directory containing the text 'with stuff in it.' i. The `find` command recursively prints all the files in a directory . Try running it on the 'this' directory. Now use `>` again to save the outputs of that command to a file called 'contents'. j. Use `mv` to move the entire 'this' directory structure into your assignment1 folder - this should only take running `mv` once. Then run `mv` again to put the 'content' file there too. Done with the core exercises! If you haven't yet, please remember to fill out the top of the file - your name, time spent, and any questions/comments. ###################### # Extra Exercises (9-10) ###################### #### 9. Text editor wars! I mentioned previously that there's a lot of controversy and differing opinions about text editors. I also mentioned that it's worth ultimately learning vim or emacs if you intend to keep programming - there's an input cost to learning but the speed and functionality you gain will pay off in the long run. Do a bit of looking into vim and emacs, and which you might prefer. One place to get some context is this Wikipedia article: https://en.wikipedia.org/wiki/Editor_war Or you can just do some googling. Or choose randomly. In any case, pick one and try doing the tutorial they provide. For emacs do: emacs Ctrl-h then 't' For vim do: vimtutor Try to get through one or both of the tutorials! Let me know which you did, and how you found it! >>> #### 10. Aliasing (Note: This isn't relevant for folks using PuTTy or MobaXTerm on Windows, so just skip it.) Here we're going to create an 'alias' which allows you to make a new command that's a shortcut for another, more complicated command, potentially with arguments. We'll do this in a 'dotfile' (see the description of dotfiles in the part of section 4 about nano) called '.bash_aliases'. Open a new terminal window on your computer (not Quest). First, make sure you're running bash. Check what shell you're running with: echo "$SHELL" If it's bash (most likely) you'll get an output like '/bin/bash'. If it's not bash, google around to either figure out a) how to set your default shell to bash (often an option in your terminal program) or b) how to make aliases in whatever shell you're using. Using your new expertise with your text editor, open the file '.bash_aliases' in your home directory. This file will likely be blank. Once you open the file, write a new line that looks like this: alias quest="ssh [netID]@quest.it.northwestern.edu" Notice how in the quotes we have the normal command we run to login to Quest. You can also change the 'quest' there to alias this command to anything you want. Save the dotfile and quit, then close and re-open your terminal. Try typing your alias, and see what happens! If it isn't working, you may need to also edit the file called ~/.bashrc to include a line that reads `source ~/.bash_aliases`. This tells bash upon opening to also look in .bash_aliases for any additional commands to run. Look here for reference: https://www.raspberrypi.org/documentation/linux/usage/bashrc.md Try making another alias that makes it so you can type 'hi' on your terminal and your terminal will reply and ask how you're doing. What line did you add to .bash_aliases? >>>