Video summary

How I Actually Use AI for Data Science in Python

Main summary

Key takeaways

Technology

Summary of technological concepts & workflow (AI for Python data science)

End-to-end workflow in Python with AI assistance

  • Load data into a Jupyter notebook inside an IDE (shown with PyCharm / JetBrains).
  • Use AI to suggest and/or generate code transformations and to help with:
    • data inspection
    • data cleaning
  • Emphasize the critical step: verification + data visualization/analysis to ensure AI changes are correct.

Data access & sources

Start with real datasets from:

  • CSV files
    • Quick viewing/sorting up front
    • Primarily handled via DataFrames in Jupyter
  • A local SQLite database
    • Notes also mention connecting to remote databases

In PyCharm:

  • Database tools allow viewing/querying schema and running SQL.
  • Jupyter can run SQL cells and save query results directly into DataFrame variables for later transformations.

Why Jupyter notebooks are used

Prefer notebooks over plain scripts/terminal due to:

  • Cell-based execution
  • Easier experimentation
  • The ability to view outputs per step while cleaning/validating data

Enabling and configuring AI inside the IDE

In PyCharm:

  • Enable the AI chat plugin/feature.
  • Choose different agents/providers and models (not locked to one provider).

Support mentioned includes:

  • Selecting higher-end vs cheaper models depending on token/budget needs
  • Using local models via PyCharm integration
  • Adding MCP servers and API keys
    • Example: GitHub MCP to create repos, sync changes, etc.

Safety modes mentioned:

  • “bypass permissions” for convenience
  • “accept edits mode” for sensitive data so the AI does not execute changes without approval

AI-driven transformations with strong guardrails

Example use case: convert a “revenue” column from strings to numeric/decimal.

Key problem: the column contains mixed currency formats:

  • US format vs European format
  • e.g., values like 9.537,48

The speaker warns AI may incorrectly “strip” symbols/commas/dots and mangle values.

Verification approach:

  • Before accepting transformations, inspect raw values and column formats
  • Ask AI to verify all revenue formats before applying conversion logic
  • Confirm results by re-checking the DataFrame values and using visualization tools

“Skills” / rules for reusable conventions

Introduces AI skills/rules to encode project-specific conventions, such as:

  • column naming in snake_case
  • reviewing columns before changes
  • safety checks

These skills are saved (example: creating a markdown skill file like conventions) so they can be reused for future tasks and keep AI behavior consistent.

Built-in DataFrame inspection & outlier handling (verification layer)

PyCharm’s DataFrame viewer provides:

  • Column statistics (off/compact/detailed)
  • Hover/type info (e.g., float64/string)
  • Sorting to find patterns/outliers
  • Visual flags (e.g., problematic values highlighted in red)

Automated remediation workflow:

  • Select detected issues (e.g., missing values / None counts, outliers, weird unit-price distributions)
  • Use “fix with AI” to generate and apply corrections
  • Re-check statistics afterward to confirm improvement

SQL + DataFrame integration

In Jupyter:

  • Run SQL queries against a chosen source (e.g., the sales SQLite DB)
  • Automatically save results into a DataFrame variable (e.g., results)

Optional PyCharm database UI features:

  • View schema
  • Edit/inspect tables without leaving the IDE
  • Provide accurate schema/table info to better inform AI prompts

Data wrangling and chart generation

Use Data Wrangler (within PyCharm) to:

  • Filter/sort
  • Find/replace values
  • View outliers
  • Create charts (chart view supported)

AI for visualization:

  • Prompt AI to generate complex Matplotlib plots
  • Let AI create code/plots while relying on IDE tools to validate/inspect output

Central takeaway

  • AI code generation is relatively easy.
  • The hard part is verification.
  • Success depends on using IDE/data tools that support:
    • interactive inspection
    • visualization
    • statistics/outlier detection
    • comparison back to authoritative sources (CSV/DB)

Main speakers / sources

  • Main speaker/source: The video creator (unnamed) demonstrating the workflow in PyCharm (JetBrains).
  • Referenced tools/brands:
    • JetBrains PyCharm
    • Jupyter Notebook
    • SQLite
    • Claude (mentioned in skills context)
    • GitHub MCP (example MCP usage)
    • WhisperFlow (voice dictation mention)

Original video