🌐 Selenium: Special Web Elements

Taming the Wild Creatures of the Web

Imagine you’re a zoo keeper, but instead of animals, you take care of special creatures on websites. Some creatures are easy to handle—like buttons and text boxes. But today, we’re going to learn how to handle the trickier creatures: Tables, Date Pickers, Auto-suggestions, and sneaky Broken Links!

🦁 The Four Special Creatures

📊 HTML Tables: The Grid Maze

What is an HTML Table?

Think of an HTML table like a chocolate box. It has rows (horizontal lines) and columns (vertical lines). Each little square is called a cell—just like each chocolate sits in its own spot!

graph TD
    A[Table] --> B[Row 1]
    A --> C[Row 2]
    A --> D[Row 3]
    B --> E[Cell 1,1]
    B --> F[Cell 1,2]
    C --> G[Cell 2,1]
    C --> H[Cell 2,2]

Why Tables Can Be Tricky

• Tables have many cells to navigate
• Data can change (new rows added)
• You need to find the right cell among many

How Selenium Finds Table Data

Step 1: Find the table

table = driver.find_element(
    By.ID, "priceTable"
)

Step 2: Get all rows

rows = table.find_elements(
    By.TAG_NAME, "tr"
)

Step 3: Get cells in a row

cells = rows[1].find_elements(
    By.TAG_NAME, "td"
)

Step 4: Read the text

price = cells[2].text
print(price)  # "$99.99"

The Magic XPath Trick

Want cell in row 3, column 2? Use this shortcut:

cell = driver.find_element(
    By.XPATH,
    "//table[@id='myTable']"
    "/tbody/tr[3]/td[2]"
)

🎯 Remember: XPath counts from 1, not 0!

Real Example: Reading a Product Table

# Find all product names
products = driver.find_elements(
    By.XPATH,
    "//table[@id='products']"
    "//td[1]"
)

for product in products:
    print(product.text)

📅 Date Pickers: The Calendar Puzzle

What is a Date Picker?

A date picker is like a mini calendar that pops up when you want to choose a date. You’ve seen it when booking movie tickets or flights!

Why Date Pickers Are Sneaky

• They pop up and disappear
• Some let you type, others don’t
• Different websites use different calendar designs

Three Ways to Handle Date Pickers

graph TD
    A[Date Picker] --> B{Can you type?}
    B -->|Yes| C[Direct Input]
    B -->|No| D{JavaScript?}
    D -->|Yes| E[JS Injection]
    D -->|No| F[Click Navigation]

Method 1: Direct Input (Easy Mode)

If the date field accepts typing:

date_field = driver.find_element(
    By.ID, "birthDate"
)
date_field.clear()
date_field.send_keys("12/25/2024")

Method 2: JavaScript Magic

Force the date value directly:

driver.execute_script(
    "document.getElementById"
    "('birthDate').value"
    "='2024-12-25'"
)

Method 3: Click Through Calendar

When you must click month/day:

# Open calendar
driver.find_element(
    By.ID, "dateBtn"
).click()

# Click next month arrow
driver.find_element(
    By.CLASS_NAME, "next-month"
).click()

# Click the day
driver.find_element(
    By.XPATH,
    "//td[@data-day='25']"
).click()

Pro Tip: Wait for Calendar

from selenium.webdriver.support.ui import (
    WebDriverWait
)
from selenium.webdriver.support import (
    expected_conditions as EC
)

# Wait for calendar to appear
WebDriverWait(driver, 10).until(
    EC.visibility_of_element_located(
        (By.CLASS_NAME, "calendar")
    )
)

🔮 Auto-suggestions: The Mind Reader

What is Auto-suggestion?

Auto-suggestion is when a website guesses what you want to type! Like when you type “Piz” in a food app and it shows “Pizza” before you finish.

The Challenge

• Suggestions appear and disappear quickly
• You must wait for them to load
• You need to click the right suggestion

How Auto-suggestion Works

graph TD
    A[You type 'sel'] --> B[Wait 0.5s]
    B --> C[Dropdown appears]
    C --> D[Shows: Selenium]
    C --> E[Shows: Select]
    C --> F[Shows: Self]
    D --> G[Click Selenium]

Step-by-Step Code

Step 1: Find the search box and type

search = driver.find_element(
    By.ID, "searchBox"
)
search.send_keys("selen")

Step 2: Wait for suggestions

WebDriverWait(driver, 5).until(
    EC.visibility_of_element_located(
        (By.CLASS_NAME, "suggestions")
    )
)

Step 3: Get all suggestions

items = driver.find_elements(
    By.CLASS_NAME, "suggestion-item"
)

Step 4: Click the one you want

for item in items:
    if "Selenium" in item.text:
        item.click()
        break

The Smart Way: Keyboard Navigation

Sometimes clicking is tricky. Use keyboard arrows!

from selenium.webdriver.common.keys import Keys

search = driver.find_element(
    By.ID, "searchBox"
)
search.send_keys("selen")

# Wait for suggestions
time.sleep(1)

# Press down arrow twice
search.send_keys(Keys.ARROW_DOWN)
search.send_keys(Keys.ARROW_DOWN)

# Press Enter to select
search.send_keys(Keys.ENTER)

🔗 Broken Links: The Dead Doors

What is a Broken Link?

A broken link is like a door that leads to nowhere. You click it, but the page doesn’t exist! It shows errors like 404 Not Found.

Why Check for Broken Links?

• Bad user experience - visitors get frustrated
• SEO problems - search engines don’t like them
• Quality assurance - professional sites fix them

How Links Break

graph TD
    A[Good Link] --> B[Page Deleted]
    B --> C[Broken Link!]
    A --> D[URL Changed]
    D --> C
    A --> E[Server Down]
    E --> C

Two Steps to Find Broken Links

Step 1: Get all links on the page

links = driver.find_elements(
    By.TAG_NAME, "a"
)

urls = []
for link in links:
    href = link.get_attribute("href")
    if href:
        urls.append(href)

Step 2: Check each link’s status

import requests

def check_link(url):
    try:
        response = requests.head(
            url, timeout=5
        )
        return response.status_code
    except:
        return 0

Complete Broken Link Checker

import requests
from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get("https://example.com")

# Get all links
links = driver.find_elements(
    By.TAG_NAME, "a"
)

broken = []
for link in links:
    url = link.get_attribute("href")
    if url and url.startswith("http"):
        code = check_link(url)
        if code >= 400 or code == 0:
            broken.append({
                "url": url,
                "status": code
            })
            print(f"BROKEN: {url}")

print(f"Found {len(broken)} broken")

Status Code Quick Guide

🎯 Summary: Your Special Elements Toolkit

🧙‍♂️ Final Wisdom

“The best Selenium tester knows that special elements need special attention. Take your time, wait for things to appear, and always have a backup plan!”

Remember our zoo keeper analogy? You’ve now learned to handle:

• 📊 Tables - Navigate the grid maze
• 📅 Date Pickers - Solve the calendar puzzle
• 🔮 Auto-suggestions - Catch the mind reader
• 🔗 Broken Links - Find the dead doors

You’re ready to tame any special web element! 🎉