What is a Library?
Essentially a list of pre-written code that you can use to streamline and clean up your program.
Libraries can help simplify complex programs
APIS are specifications for how the procedures in a library behave, and how they can be used
Documentations for an API/library is necessary in understanding the behaviors provided by the API/library and how to use them
Libraries that we will go over: Requests, Pillow, Pandas, Numpy, Scikit-Learn, TensorFlow, matplotlib.
Required Installations
Please run the following commands in your vscode terminal in order to continue the lesson
- pip install numpy
- pip install matplotlib
- pip install scikit-learn
- pip install pillow
- pip install pandas
- pip install tensorflow
- pip install requests
Please run the following commands in your vscode terminal in order to continue the lesson
- pip install numpy
- pip install matplotlib
- pip install scikit-learn
- pip install pillow
- pip install pandas
- pip install tensorflow
- pip install requests
Images using requests and pillow libraries
‘Requests’ is focused on handling HTTP requests and web data while ‘Pillow’ is designed for data manipulation and analysis It’s common to see them used together in data-related assignments where data is fetched by HTTP requests using Requests and then processed and analyzed with Pandas.
Here’s an example:
import requests
from PIL import Image
from io import BytesIO
# Step 1: Download an image using Requests
image_url = "https://example.com/path/to/your/image.jpg" # Replace with the actual URL of the image you want to download
response = requests.get(image_url)
if response.status_code == 200:
# Step 2: Process the downloaded image using Pillow
image_data = BytesIO(response.content) # Create an in-memory binary stream from the response content
img = Image.open(image_data) # Open the image using Pillow
# Perform image processing tasks here, like resizing or applying filters
img = img.resize((x, y)) # Resize the image and replace x,y with desired amounts
# Step 3: Save the processed image using Pillow
img.save("processed_image.jpg") # Save the processed image to a file
print("Image downloaded, processed, and saved.")
else:
print(f"Failed to download image. Status code: {response.status_code}")
Failed to download image. Status code: 404
In this code, we use the Requests library to download an image from a URL and then if the download is successful the HTTP status code 200 will pop up, and from there we create an in-memory binary stream (BytesIO) from the response content. We then use the Pillow library to open the image, make any necessary changes, and save the processed image to a file.
Here’s a step by step tutorial on how we wrote this code: 1)We started by importing the necessary libraries, which were Requests, Pillow, and io.
2)Download the Image
3)Use the Requests library to send an HTTP GET request to the URL to download the image. Check the response status code to make sure the download goes through(status code 200).
4)If the download is successful, create an in-memory binary stream (BytesIO) from the response content. Process the Image:
5)Utilize the Pillow library to open the image from the binary stream. Change photo to desired preference(ie: size) Save the Processed Image:
6)Save the processed image to a file using Pillow. Choose a filename and file format for the saved image.
Hack 1
Write a Python code that accomplishes the following tasks:
Downloads an image from a specified URL using the Requests library. Processes the downloaded image (like resizing) using the Pillow library. Save the processed image to a file.
import requests
from PIL import Image
from io import BytesIO
# Step 1: Download a random image using Requests
image_url = "https://source.unsplash.com/random" # Use a source that provides random images
response = requests.get(image_url)
if response.status_code == 200:
# Step 2: Process the downloaded image using Pillow
image_data = BytesIO(response.content) # Create an in-memory binary stream from the response content
img = Image.open(image_data) # Open the image using Pillow
# Perform image processing tasks here, like resizing or applying filters
img = img.resize((200, 400)) # Resize the image and replace with desired dimensions
img.save("processed_image.jpg") # Save the processed image to a file
print("Random image downloaded and saved.")
else:
print(f"Failed to download image. Status code: {response.status_code}")
Random image downloaded and saved.
Math Operations With Python Libraries
Numpy(Numerical Python) is used for numerical and scientific computing. It provides tools for handling large sets of numbers, such as data tables and arrays. Numpy makes it easier and more efficient to do mathematical tasks.
The Matplotlib library lets you create a visual representation of your data (graphs, charts, and etc.)
Example Sine Graph
Uses numpy and matplotlib libaries
import numpy as np
import matplotlib.pyplot as plt
# Generate sample data with NumPy
x = np.linspace(0, 2 * np.pi, 100)
# Create an array of values from 0 to 2*pi
# 100 is included to have 100 points distributed between 0 and 2π to make graph smoother
y = np.sin(x)
# Compute the sine of each value
# Create a simple line plot using Matplotlib
plt.plot(x, y, label='Sine Function', color='blue', linestyle='-') # Create the plot
plt.title('Sine Function') # Set the title
plt.xlabel('x') # Label for the x-axis
plt.ylabel('sin(x)') # Label for the y-axis
plt.grid(True) # Display a grid
plt.legend() # Show the legend
plt.show() # Display the plot
Hack 2
Using the data from the numpy library, create a visual graph using different matplotlib functions.
import numpy as np
import matplotlib.pyplot as plt
# Generate data for two lines
x = np.linspace(0, 10, 50) # Create an array of values from 0 to 10
y1 = 2 * x + 1 # Set of data poits
# Create and display a plot using Matplotlib
# your code here
plt.plot(x, y1, label='Line Function', color='red', linestyle='-') # Create the plot
plt.title('Line Function') # Set the title
plt.xlabel('x') # Label for the x-axis
plt.ylabel('y') # Label for the y-axis
plt.grid(True) # Display a grid
plt.legend() # Show the legend
plt.show() # Display the plot
Tensor Flow is used in deep learning and neural networks, while scikit-learn is used for typical machine learning tasks. When used together, they can tackle machine learning projects. In the code below, Tensor Flow is used for model creation and training. Scikit-learn is used for data-processing and model evaluation.
Pip install tensorflow scikit-learn
import numpy as np
import tensorflow as tf
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.preprocessing import StandardScaler
from tensorflow import keras
from tensorflow.keras import layers
# Generate synthetic data
np.random.seed(0)
X = np.random.rand(100, 1) # Feature
y = 2 * X + 1 + 0.1 * np.random.randn(100, 1) # Target variable with noise
# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Standardize the features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# Create a simple linear regression model using TensorFlow and Keras
model = keras.Sequential([
layers.Input(shape=(1,)),
layers.Dense(1)
])
# Compile the model
model.compile(optimizer='adam', loss='mean_squared_error')
# Train the model
model.fit(X_train, y_train, epochs=100, batch_size=32, verbose=2)
# Make predictions on the test set
y_pred = model.predict(X_test)
# Calculate the Mean Squared Error on the test set
mse = mean_squared_error(y_test, y_pred)
print(f"Mean Squared Error: {mse:.4f}")
A decrease in loss and time metrics (ms/epoch and ms/step) shows the efficiency increases as the training epochs increases
Hack
fill in the missing code to match the custom data set
# Import necessary libraries
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_squared_error
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
# Assuming you have X and y defined, your data
# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
# Standardize the features
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)
# Create a regression model using TensorFlow and Keras
model = keras.Sequential()
model.add(layers.Dense(10, input_shape=(X_train.shape[1],), activation='relu')) # Adjust input shape to match the number of features
model.add(layers.Dense(1, activation='linear'))
# Compile the model for regression
model.compile(optimizer='adam', loss='mean_squared_error') # Use 'mean_squared_error' for the loss
# Train the model
model.fit(X_train, y_train, epochs=100)
# Make predictions on the test set
y_pred = model.predict(X_test)
# Calculate the Mean Squared Error on the test set
mse = mean_squared_error(y_test, y_pred)
print(f"Mean Squared Error on Test Set: {mse:.2f}")
/usr/lib/python3/dist-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.26.1
warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
2023-10-26 17:34:56.876652: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-10-26 17:34:58.111785: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2023-10-26 17:34:58.111839: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2023-10-26 17:34:58.117128: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-10-26 17:34:58.799744: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-10-26 17:34:58.808703: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-10-26 17:35:02.755505: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
/tmp/ipykernel_2450/1101574954.py in <module>
11
12 # Split the data into training and testing sets
---> 13 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
14
15 # Standardize the features
NameError: name 'X' is not defined
HOMEWORK 1
Create a GPA calculator using Pandas and Matplot libraries and make: 1) A dataframe 2) A specified dictionary 3) and a print function that outputs the final GPA
Extra points can be earned with creativity.
import pandas as pd
import matplotlib.pyplot as plt
#class numbers
classcount = int(input("Enter the number of classes: "))
#List to store the grades
grades = []
#User input + data frame and dicitonary
for i in range(1, classcount + 1):
grade = input(f"Enter the grade for Class {i} (A, B, C, D, F): ")
grades.append(grade)
grades_data = pd.DataFrame({'Grade': grades})
grade_points = {'A': 4, 'B': 3, 'C': 2, 'D': 1, 'F': 0}
#calculate
grades_data['Points'] = grades_data['Grade'].map(grade_points)
gpa = grades_data['Points'].mean()
#printer
def print_final_gpa():
print("\nGrades Entered:")
print(grades_data)
print("\nYour GPA is: {:.2f}".format(gpa))
#Display the bar plot of grades
grades_data['Points'].plot(kind='bar', color='skyblue')
plt.xlabel('Classes')
plt.ylabel('Grade Points')
plt.title('Grades for Each Class')
plt.xticks(ticks=range(len(grades_data)), labels=[f"Class {i}" for i in range(1, classcount + 1)])
plt.tight_layout()
plt.show()
#Call the function to print the final GPA
print_final_gpa()
Grades Entered:
Grade Points
0 A 4
1 A 4
2 A 4
3 B 3
4 A 4
Your GPA is: 3.80
HOMEWORK 2
Import and use the “random” library to generate 50 different points from the range 0-100, then display the randomized data using a scatter plot.
Extra points can be earned with creativity.
import matplotlib.pyplot as plt
import random
#random x and y coordinates
x = [random.randint(0, 100) for _ in range(50)]
y = [random.randint(0, 100) for _ in range(50)]
#random color palette (da creativity)
colors = [random.choice(['#FF5733', '#33FF57', '#3388FF', '#A033FF', '#FF3388']) for _ in range(50)]
#random point selection using markers
markers = ['o', 's', '^', 'D', 'x'] * 10
#scatter plot
plt.figure(figsize=(8, 6))
plt.scatter(x, y, c=colors, s=100, marker=random.choice(markers), edgecolors='black', alpha=0.7)
#display and labels
plt.title('Randomized Data Scatter Plot', fontsize=16)
plt.xlabel('X-axis', fontsize=12)
plt.ylabel('Y-axis', fontsize=12)
plt.grid(True)
plt.tight_layout()
plt.show()
