Skip to content

Latest commit

 

History

History
308 lines (231 loc) · 12.6 KB

Aug_22_2019.md

File metadata and controls

308 lines (231 loc) · 12.6 KB

August 22, 2019
ATMOS 5020: Environmental Programming
Brian Blaylock and John Horel

Today's Objectives

  1. Introduce concept of programming languages
  2. Review basic Linux commands and learn some new ones
  3. Move files between the classroom Mac and CHPC cluster and access the file on the internet.
  4. Edit and run short shell scripts.



Computer Binary

Computers know only about strings of 0 and 1

  • Bits: Binary digITs (two bits: 0 or 1)
  • Byte: 8 bits
  • 32 or 64 bit words
  • Combinations of bytes and words often expressed as hexidecimals (base 16)
  • Computers count with bits; humans use base 10
  • Computer program is required to translate from/to base 10

Numbers as bits

Integer Bit String Equation
0 0 (0 x 20)
1 1 (1 x 20)
2 10 (1 x 21) + (0 x 20)
3 11 (1 x 21) + (1 x 20)
4 100 (1 x 22) + (0 x 21) + (0 x 20)
5 101 (1 x 22) + (0 x 21) + (1 x 20)
6 110 (1 x 22) + (1 x 21) + (0 x 20)
7 ? ?
8 ? ?
9 ? ?
10 1010 (1 x 23) + (0 x 22) + (1 x 21) + (0 x 20)
16 10000 24
32 100000 ?
64 1000000 26
128 10000000 ?
256 100000000 28

Online Binary Converter Tool

  • What number is defined by a string of 16 1’s? 32 1’s? 64 1’s?

ASCII Character Codes

Languages tend to use a similar syntax. Characters are represented by 8 bit (1 byte) strings defined by the American Standard Code for Information Interchange. http://www.ascii-code.com/

Online ASCII Character to Binary Converter

  • How do you spell your name in Binary?
Word Binary Binary Values
Dog 01000100 01101111 01100111 68 111 103
Cat 01000011 01100001 01110100 67 97 116
python 01110000 01111001 01110100 01101000 01101111 01101110 112 121 116 104 111 110

How do you know whether a bit string represents a number or a character?

This is interpreted by the program you are using. Each language defines syntax for different types of “variables” or “data types” e.g. character, integer, float.

A variable of data type “character” can be assigned a length so that a 20 character variable is 20 characters * 8 bits/character= 160 bits

  • Some languages require you to "declare" the variable's type (Fortran, C, etc.).
  • In Python, variable declaration is dynamic and the program will decide for you. (This is a convenient advantage of Python)
    • 10 is an integer. Binary: 1010
    • 10.625 is a float. Binary: 1010.101
    • "10.3" is a character. Binary (ascii) 00110001 00110000 00101110 00110011



Programming languages

Programming is a way to pass a list of instructions to a computer. Each language has its own syntax (form) and semantics (meaning).

  • Syntax: Sequences of text including words, numbers, and punctuation using rules like written languages. The "grammar rules" for that language.
  • Semantics: The meaning given to the syntax - a sequence of words that makes sense to a computer. The symbol ! or ^ can mean different things in different languages.

Compiled vs. Interpreted Languages

Compiled

  • Translates the language syntax (source code) to machine code (binary) before executing the program.
  • Create a binary executable that is “run”
  • C and Fortran are examples of compiled languages
  • Compiled codes typically take less time to complete
  • Usually requires recompiling on different platforms: Linux vs. PC

Interpreted

  • Executes source code directly translating statements using code snippets already compiled in machine language.
  • Create and run a script that executes instructions
  • Java, Python, Perl, and Julia are examples of interpreted languages
  • Tend to be slower but also more platform independent. (You can run python code you write on the Mac on your PC as long as you have the correct Python interpreter installed).





Linux commands (review)

Perhaps one of the most repeated tasks in programing is to get data files, move data files, copy data files, remove data files, change data files, etc. That is why we are spending so much time learning how to move and manipulate files in Linux (you want to develop the muscle memory).

Linux commands follow the pattern

<command> <options or "flags"> <directory or file>

For example, to list all the files in a specific directory and to show more information about those files...

ls ./myDirectory/
ls -l ./myDirectory/

Copy and move files
cp file1 file2 copies file1 to file2
cp file1 dest_directory/ copy file1 to the specified existing directory
cp * dest_directory/ Copy all files in current directory to dest
cp -r directory1 dest_directory/ Recursively copies files from directory1 to the dest_directory
mv file1 new-file renames file1 as new-file
mv file1 dest_directory/ moves file1 to the specified existing directory

Removing files and directories

There is no undelete! Be careful!!!
rm fileName Remove a file. Cannot be undone!
rm -i file Interactive removal (option –i) of specified file. User must answer yes or no before deletion.
rm -r directoryName Don't DO THIS! Unless you know what you are doing. A recursive deletion of all files within the specified directories and the directory

Path Navigation
pwd Tells you were you are at.
/ Separates directories
. refers to the current directory. Example: cp /this/File.txt ./
.. refers to parent directory (back one). Example: cd ../../myDir
* Wildcard. For example, list all files ending in .txt: ls *.txt
~ Refer to your home directory. Example: ~/public_html/ take you to your public_html directory no matter what your current path is.

Pro Tips:

  1. Use the up and down arrows to look at old commands.
  2. Use the tap key to auto-complete directory or file names that exist.



Linux User Permissions

Linux machines allow multiple people to use them. Permissions define who has access to the files.

Linux distinguishes permission for three different types.

  • u: User This is "you." (Type whoami to see who you are)
  • g: Group The group is anyone related to your organization (Check with groups).
  • o: Others This is anyone with access to the Linux system.

Each type can have privileges to read (r) write (w) and execute (x).

  • r: Read privileges. Can you see the contents of the file.
  • w: Write privileges. Can you edit the contents of the file.
  • x: Execute privileges. Can you use the file to execute commands or run a program script?

Check permissions with ls –l

  • drwxrwxrwx (directory, user-group-other all have read, write, and execute privileges)
  • -rwx------ (file and only the user can rwx the file)
  • -r--r--r-- (file with only read permissions for everyone)

Use the chmod ("change mode") command to change file permission.

Examples
chmod +x fileName Gives yourself execute permissions on a file
chmod ugo+rwx fileName Gives everyone (user, group and others) permission to read, write, and execute file
chmod o-wx fileName Remove permission by all others to write or execute a file.



Exercises

E1: I want to...

  1. Open a terminal window using Xquartz or Terminal application
  2. List the files in my Desktop folder
    • Do you use cd, mv, or ls?
  3. Make a subdirectory on the Desktop named aug_22
    • Do you use cd, ls, mkdir, or cp?

E2: I want to...

  1. Enter the aug_22 directory you just created.
  2. Copy an image from the internet to your home directory
    • curl -O https://pando-rgw01.chpc.utah.edu/GOES16/ABI-L2-MCMIPC/20190822/OR_ABI-L2-MCMIPC-M6_G16_s20192341846184_e20192341848569_c20192341849111.png
    • (that option -O is a capital O, not a zero)
  3. What is this an image of?
  4. Make a copy of the image and rename it with a much shorter name.
  5. How do you find the time each image was created?
  6. Remove the old image.

E3: I want to...

  1. Return to my home directory
    • what command is simplest to use?
  2. List the subdirectories (and files) there and save them to a file named files.txt
    • how do you do that?
  3. Move the file to my aug_22 directory (you can either copy or move them- what’s the difference?) and give the file a new name files_2.txt
    • how do you do that?
  4. Verify that the file is in that directory and see the date the file was modified and have the files sorted in reverse order
    • how do you do that? There are a couple of ways to do that
    • Try ls -r > files_2.txt
    • Try ls > files_3.txt
    • Try sort files_3.txt
    • Try sort -r files_3.txt

E4: I want to…

  1. See who is logged onto the computer
    • who
  2. See a list of all the commands I’ve typed
    • history
  3. Return the current date and time
    • date (The local date of the machine)
    • date -u (This the date in UTC time)
  4. Repeat the last command
    • !!




Now we are going to access a different computer

The Center for High-Performance Computing (CHPC) offers many resources not available on these classroom computers. This section will show you how to access CHPC resources. Today you will create a new directory accessible on the CHPC web server.

Log onto a CHPC computer: meteo07

In the terminal, type the following:

ssh –l uXXXXXXX –Y meteo07.chpc.utah.edu
  • ssh means to secure login to a remote computer.
  • -l specifies the "user" option (replace the Xs with your unid)
  • The –Y option allows secure x11 forwarding for graphics

NOTE: When you type your password, the cursor won't move or type *, it will stay blank. If you think you messed up, hold the backspace key for a while then start again. Then type yes when/if prompted to do so (first time you log in).

(You may also log into meteo08.chpc.utah.edu)

You just logged onto a different computer located at the University's Downtown Data Center. This is a Linux system.

Type pwd to see where you are at.

👀 TRY THIS! Type xeyes and hit enter. Do a pair of eyes pop up? This is a quick test to see if X11 is working. X11 enables you to see graphics on your screen from the remote computer.

Make a Public HTML Directory

  1. Make a new directory called public_html.
    • mkdir public_html
  2. Go to that directory
    • cd public_html
  3. Make a directory called 5020 and cd into it.
    • mkdir 5020
    • cd 5020
  4. Copy a file of precipitation data from Utah
    • cp ~u0035056/public_html/5020/utah_precip_oct.csv .
      • the . at the end indicates the directory I am currently in
      • The syntax you are using to copy the file: cp [file from here] [put it here]
    • What did you just do?
  5. Look in your new web directory using a browser
    • http://home.chpc.utah.edu/~uXXXXXXX/
      • Replace the X’s with your unid
    • What is there now in your directory?
    • Files you put in this public_html directory--like images, HTML, etc.--are accessible by you and others by the web.

🤚 "I need a reminder of how to log onto the CHPC computer!" Follow the instructions above and follow this YouTube tutorial, Logging Onto CHPC resources: https://www.youtube.com/watch?v=hP2GDWCTKg4

🤚 How do I log onto CHPC resources on my Windows PC? Use Putty and Xming.

E5: I want to

  1. Look at the precipitation data from Utah from Oct 1-9, 2018
    • more utah_precip_oct.csv
    • type q or ctrl-c to exit out of more.
  2. Sort the file so that the highest values are at the top and save the output to utah_precip_oct_sort.csv
    • sort –r utah_precip_oct.csv > utah_precip_oct_sort.csv
      • Is it sorted correctly?
      • What does the –r modifier do?
  • Try this:
    • sort –r -g utah_precip_oct.csv > utah_precip_oct_sort.csv
    • Is that better? Why?
    • Do a man sort to read the documents.




What should you be doing?

  • Review all the material in these notes.
  • Practice!
  1. Check your understanding #1 (due August 23)
  2. Check your understanding #2 (due August 30)