Excel Processing

by egany

data

Best practices for robust Excel data processing with Pandas and OpenPyXL

Skill Details

Repository Files

1 file in this skill directory


name: excel-processing description: Best practices for robust Excel data processing with Pandas and OpenPyXL

Excel Processing Skill

Guide for efficient, safe, and standards-compliant Excel data processing.

📖 Reading Excel Files

1. Engine Selection

Always determine the appropriate engine:

  • .xlsx: Use openpyxl (Default modern format).
  • .xls: Use xlrd (Legacy format).
  • .csv: Use pandas.read_csv.

2. Robust Reading Pattern

def read_excel_safe(filepath):
    try:
        if filepath.lower().endswith('.xlsx'):
            return pd.read_csv(filepath, engine='openpyxl')
        elif filepath.lower().endswith('.xls'):
            return pd.read_csv(filepath, engine='xlrd')
        return None
    except Exception as e:
        print(f"Error: {e}")
        return None
    ```

### 3. Handling Temp Files
Always skip Excel temp files (`~$filename.xlsx`):
```python
if filename.startswith('~$'):
    continue

💾 Writing Excel Files

1. Preserving Data

Use index=False unless index has meaning:

df.to_excel("output.xlsx", index=False, engine='openpyxl')

2. Large Data Sets

For large datasets (>100k rows), openpyxl can be slow. Consider:

  • Splitting into multiple files.
  • Using CSV if formatting is not needed.

🛡️ Error Handling Patterns

1. File Locking

Excel file open by user will be locked. Solution: Catch PermissionError.

try:
    df.to_excel("output.xlsx")
except PermissionError:
    print("Error: File is open. Please close Excel and try again.")

2. Corrupted Files

File downloaded from internet or corrupted format. Solution: Catch BadZipFile or ValueError.

3. Encoding (CSV)

CSV might have encoding issues (e.g. non-ASCII characters). Solution: Try list of common encodings.

encodings = ['utf-8', 'utf-8-sig', 'cp1252', 'latin1']
for enc in encodings:
    try:
        return pd.read_csv(file, encoding=enc)
    except:
        continue

🔎 Data Validation

Check Empty

if df.empty:
    print("File has no data")
    return

Check Columns

Ensure input file has required columns:

required = ['Name', 'Email']
if not all(col in df.columns for col in required):
    print("Missing required columns")

🚀 Performance Tips

  1. Read specific columns: pd.read_excel(..., usecols=['A', 'B']) to reduce RAM usage.
  2. Specify dtypes: dtype={'Phone': str} to avoid losing leading zeros.
  3. Process chunking: For huge files (GB), read by chunk (mostly with CSV).

✅ Checklist

  • Select correct engine (openpyxl vs xlrd)
  • Skip temp files ~$
  • Handle PermissionError (File locked)
  • Handle UnicodeDecodeError (Encoding)
  • Check df.empty before processing

🤖 Agentic Protocol

Skill Metadata

  • Version: 1.0.0
  • Last Updated: 2026-01-27

1. Activation Log

When activating this skill, print: "🎯 [SKILL ACTIVATED] excel-processing v1.0.0" "📋 Parameters:" " - Input: [file_path]" " - Operation: [read|write|validate]" " - Expected Rows: [count_if_known]"

2. User Confirmation

Before writing/modifying files: "I'll use excel-processing to [action] on [file]. Proceed? [Y/n]"

3. Completion Log

  • Success: "✅ [excel-processing] Processed [row_count] rows in [time]s"
  • Error: "❌ [excel-processing] Error: [message]"
  • Warning: "⚠️ [excel-processing] Warning: [message]"

Related Skills

Xlsx

Comprehensive spreadsheet creation, editing, and analysis with support for formulas, formatting, data analysis, and visualization. When Claude needs to work with spreadsheets (.xlsx, .xlsm, .csv, .tsv, etc) for: (1) Creating new spreadsheets with formulas and formatting, (2) Reading or analyzing data, (3) Modify existing spreadsheets while preserving formulas, (4) Data analysis and visualization in spreadsheets, or (5) Recalculating formulas

data

Clickhouse Io

ClickHouse database patterns, query optimization, analytics, and data engineering best practices for high-performance analytical workloads.

datacli

Clickhouse Io

ClickHouse database patterns, query optimization, analytics, and data engineering best practices for high-performance analytical workloads.

datacli

Analyzing Financial Statements

This skill calculates key financial ratios and metrics from financial statement data for investment analysis

data

Data Storytelling

Transform data into compelling narratives using visualization, context, and persuasive structure. Use when presenting analytics to stakeholders, creating data reports, or building executive presentations.

data

Kpi Dashboard Design

Design effective KPI dashboards with metrics selection, visualization best practices, and real-time monitoring patterns. Use when building business dashboards, selecting metrics, or designing data visualization layouts.

designdata

Dbt Transformation Patterns

Master dbt (data build tool) for analytics engineering with model organization, testing, documentation, and incremental strategies. Use when building data transformations, creating data models, or implementing analytics engineering best practices.

testingdocumenttool

Sql Optimization Patterns

Master SQL query optimization, indexing strategies, and EXPLAIN analysis to dramatically improve database performance and eliminate slow queries. Use when debugging slow queries, designing database schemas, or optimizing application performance.

designdata

Anndata

This skill should be used when working with annotated data matrices in Python, particularly for single-cell genomics analysis, managing experimental measurements with metadata, or handling large-scale biological datasets. Use when tasks involve AnnData objects, h5ad files, single-cell RNA-seq data, or integration with scanpy/scverse tools.

arttooldata

Xlsx

Spreadsheet toolkit (.xlsx/.csv). Create/edit with formulas/formatting, analyze data, visualization, recalculate formulas, for spreadsheet processing and analysis.

tooldata

Skill Information

Category:Data
Last Updated:1/27/2026