Numerical Methods

🎯 R Numerical Methods: Finding the Best, Filling the Gaps, and Smoothing the Bumps

The Treasure Hunt Analogy 🗺️

Imagine you’re on a treasure hunt. You have a map, but it’s incomplete. Numerical methods in R are your magical tools to:

1. Find the treasure (Optimization) → Where is the best spot?
2. Fill in missing map pieces (Interpolation) → What’s between the X marks?
3. Smooth out the wrinkles (Smoothing) → Make the path clearer!

Let’s go on this adventure together!

🏆 Part 1: Optimization Functions

Finding the Best Treasure Spot

What is Optimization?

Think of a mountain climber. They want to find the highest peak (maximum) or the deepest valley (minimum). That’s optimization!

Real life examples:

• Finding the cheapest flight ✈️
• The fastest route home 🏠
• The best recipe mix 🍰

The optimize() Function

This is R’s treasure finder for one-dimensional problems (like finding the lowest point on a single road).

# Find minimum of x² - 4x + 4
# (Like finding the lowest valley)
result <- optimize(
  f = function(x) x^2 - 4*x + 4,
  interval = c(-10, 10)
)

print(result$minimum)  # 2
print(result$objective) # 0

What happened?

• We told R: “Search between -10 and 10”
• R found x = 2 gives the smallest value
• The minimum value is 0

The optim() Function

When you have multiple directions to search (like finding treasure on a 2D map), use optim().

# Find minimum of (x-3)² + (y-2)²
# Like finding a valley in mountains
fn <- function(p) {
  (p[1] - 3)^2 + (p[2] - 2)^2
}

result <- optim(
  par = c(0, 0),  # Start here
  fn = fn
)

print(result$par)  # [3, 2]

The journey:

1. Start at (0, 0)
2. R walks around, always going downhill
3. Arrives at (3, 2) - the lowest point!

Optimization Methods in R

graph TD
    A["Need to Optimize?"] --> B{One Variable?}
    B -->|Yes| C["optimize"]
    B -->|No| D{Have Constraints?}
    D -->|No| E["optim"]
    D -->|Yes| F["constrOptim"]
    E --> G["Methods:"]
    G --> H["Nelder-Mead"]
    G --> I["BFGS"]
    G --> J["L-BFGS-B"]

Quick Method Guide

Practical Example: Fitting a Line

# Find best line through points
x <- c(1, 2, 3, 4, 5)
y <- c(2.1, 4.0, 5.9, 8.1, 10.0)

# We want: y = a + b*x
loss <- function(p) {
  predictions <- p[1] + p[2] * x
  sum((y - predictions)^2)
}

fit <- optim(c(0, 0), loss)
cat("Best line: y =",
    round(fit$par[1], 2), "+",
    round(fit$par[2], 2), "* x")
# Output: y = 0.02 + 1.99 * x

🧩 Part 2: Interpolation

Filling in the Missing Pieces

What is Interpolation?

You measured temperature at 9 AM (20°C) and 11 AM (26°C). What was it at 10 AM?

Interpolation connects the dots!

Linear Interpolation: approx()

The simplest approach - draw straight lines between points.

# Known points
x <- c(1, 3, 5, 7)
y <- c(10, 30, 25, 40)

# Find value at x = 4
result <- approx(
  x = x,
  y = y,
  xout = 4
)

print(result$y)  # 27.5

How it works:

• Between x=3 (y=30) and x=5 (y=25)
• Draw a straight line
• At x=4, read the value: 27.5

Smooth Interpolation: spline()

Sometimes straight lines are too rigid. Splines make smooth curves.

x <- c(1, 2, 3, 4, 5)
y <- c(1, 4, 3, 5, 2)

# Create smooth curve
smooth <- spline(
  x = x,
  y = y,
  n = 50  # 50 points
)

# Now plot it!
plot(x, y, pch = 19, cex = 2)
lines(smooth, col = "blue", lwd = 2)

Comparison: Linear vs Spline

graph TD
    A["Your Data Points"] --> B{Need Smoothness?}
    B -->|No, simple is fine| C["approx - Linear"]
    B -->|Yes, curves needed| D["spline - Smooth"]
    C --> E["Fast, simple"]
    C --> F["May look jagged"]
    D --> G["Beautiful curves"]
    D --> H["Bit more complex"]

The approxfun() and splinefun() Magic

These create reusable functions from your data!

# Create a function from data
x <- c(0, 1, 2, 3, 4)
y <- c(0, 1, 4, 9, 16)  # y = x²

my_approx <- approxfun(x, y)
my_spline <- splinefun(x, y)

# Now use like any function!
my_approx(2.5)  # Linear: 6.5
my_spline(2.5)  # Spline: 6.25 (closer!)

Real Example: Temperature Prediction

hours <- c(6, 9, 12, 15, 18, 21)
temp <- c(15, 18, 28, 30, 25, 18)

temp_func <- splinefun(hours, temp)

# What's the temperature at 10:30?
temp_func(10.5)  # ~23.7 degrees!

🌊 Part 3: Smoothing Methods

Making the Bumpy Road Smooth

Why Smooth Data?

Real data is messy. It has:

• Random noise 📻
• Measurement errors 📏
• Tiny wobbles we don’t care about

Smoothing reveals the true pattern.

Moving Average: The Simple Smoother

Like averaging grades over a semester:

# Noisy data
data <- c(10, 12, 11, 15, 14,
          13, 17, 16, 18, 20)

# Simple moving average (window=3)
smooth <- filter(
  data,
  rep(1/3, 3),
  sides = 2
)

print(round(smooth, 1))

What happened:

• Each new point = average of 3 neighbors
• Bumps get smaller
• Trend becomes clearer

Loess/Lowess: The Curvy Smoother

This is R’s most popular smoother!

x <- 1:50
y <- sin(x/5) + rnorm(50, 0, 0.3)

# Smooth it!
smooth <- loess(y ~ x, span = 0.3)

# Plot original and smooth
plot(x, y, pch = 16, col = "gray")
lines(x, predict(smooth),
      col = "red", lwd = 3)

The span Parameter

Think of span as a zoom level:

# Try different spans
par(mfrow = c(1, 3))
for (s in c(0.2, 0.5, 0.8)) {
  plot(x, y, main = paste("span =", s))
  lines(predict(loess(y~x, span=s)),
        col = "red", lwd = 2)
}

Kernel Smoothing: ksmooth()

Another smoothing option with a “kernel” (weighting pattern):

x <- 1:100
y <- sin(x/10) + rnorm(100, 0, 0.5)

# Kernel smooth
smooth <- ksmooth(
  x, y,
  kernel = "normal",
  bandwidth = 5
)

plot(x, y, col = "gray")
lines(smooth, col = "blue", lwd = 2)

Smoothing Decision Tree

graph TD
    A["Noisy Data"] --> B{What do you need?}
    B -->|Simple, fast| C["filter - Moving Avg"]
    B -->|Flexible curves| D["loess"]
    B -->|Statistical| E["ksmooth"]
    D --> F["Adjust span"]
    F -->|More detail| G["Lower span"]
    F -->|Smoother| H["Higher span"]

Comparing Smoothers

set.seed(42)
x <- 1:100
true_signal <- sin(x/10) * 10
noise <- rnorm(100, 0, 3)
y <- true_signal + noise

# Three smoothers
ma <- filter(y, rep(1/7, 7), sides=2)
lo <- predict(loess(y ~ x, span=0.3))
ks <- ksmooth(x, y, bandwidth=5)$y

# Compare!
plot(x, y, col="gray", pch=16)
lines(x, ma, col="red", lwd=2)
lines(x, lo, col="blue", lwd=2)
lines(x, ks, col="green", lwd=2)
legend("topright",
  c("Moving Avg", "Loess", "Kernel"),
  col = c("red", "blue", "green"),
  lwd = 2)

🎁 Putting It All Together

When to Use What?

Your Numerical Methods Toolkit

graph TD
    A["R Numerical Methods"] --> B["🏆 Optimization"]
    A --> C["🧩 Interpolation"]
    A --> D["🌊 Smoothing"]
    B --> B1["optimize - 1D"]
    B --> B2["optim - Multi-D"]
    C --> C1["approx - Linear"]
    C --> C2["spline - Smooth"]
    D --> D1["filter - Moving Avg"]
    D --> D2["loess - Flexible"]
    D --> D3["ksmooth - Kernel"]

💡 Key Takeaways

1. Optimization = Finding the best value

   • optimize() for single variables
   • optim() for multiple variables

2. Interpolation = Connecting dots

   • approx() for straight lines
   • spline() for smooth curves

3. Smoothing = Removing noise

   • loess() is your best friend
   • Adjust span to control smoothness

You’re now ready to find treasures, fill gaps, and smooth any bumpy data road! 🚀