Video summary

AI Engineer Reveals the Future of AI | Ex Samsung R&D on LLMs, RAG, and Careers ft. Ishan Katoch

Main summary

Key takeaways

News and Commentary

Summary of the video’s main points

The podcast features Ishaan (AI Engineer, ex-Samsung R&D) discussing his journey into AI, how the industry is evolving (LLMs, RAG, RAG evaluation, LLM vs SLM), and practical career/project advice for students. He also shares how AI work is evaluated in industry settings and what companies care about beyond just “accuracy.”

1) Ishaan’s background and transition into AI

  • Studied B.Tech CSE (Thapar Institute). Built early public-speaking/leadership experience via student societies (Toastmasters/Mudra).
  • Started exploring hackathons and won multiple events (including All-India competitions).
  • During campus placements, he realized he preferred AI/ML over traditional software roles because it felt more aligned with automation and building intelligent systems.
  • Selected for an internship at Amazon, then later got full-time conversion (PPO) as an AI Engineer.
  • Eventually Samsung Prism (a remote internship during college) helped him gain research/ML mentorship experience—working on prototype work involving Samsung’s voice assistant–type tech (Bix), with literature-style reporting and reviews.

2) How industry hiring and progression differs for AI vs pure coding

He notes hiring often hinges on:

  • Technical depth plus the ability to explain business/impact

Not just “DSA and coding,” but how you measure results and justify choices.

He claims many candidates get stuck when they can’t articulate:

  • Train/test setup, evaluation logic, and why results are high
  • Their own real contribution vs “copied” or AI-generated work

3) Career roadmap for students entering AI

Ishaan recommends a staged approach:

  • First year: explore widely (full-stack, front-end, back-end, ML/NLP), attend events, do public speaking, and try different domains. Don’t “lock in” too early.
  • Second year: maintain GPA (he suggests aiming roughly 8.5–9+ in his context), then focus on what you like and start competitive programming/events if relevant.
  • Third/Fourth year: connect with professionals, build deeper specialization, and pursue internships/projects aligned to AI/ML.

Key emphasis:

  • Projects matter more than buzzwords.
  • You must be able to explain architecture, debugging, trade-offs, scalability, and production constraints.
  • Documentation is crucial in industry (so others can take over and do KT—knowledge transfer).

4) What “good AI projects” mean (production readiness)

He argues production readiness requires more than a working demo:

  • security, reliability, error handling, performance considerations
  • iterative fixing of failures and edge cases

He stresses that interviewers will probe beyond the resume:

  • “If you used Docker/AWS/GCP, do you understand the concepts?”
  • “What exactly did you do in the project?”

5) Explaining RAG and AI evaluation metrics

He introduces RAG (Retrieval-Augmented Generation) as:

  • a pipeline where documents are retrieved, then reranked, then fed to an LLM to answer groundedly.

RAG helps reduce hallucinations by grounding responses in provided documents.

Evaluation and benchmarking:

  • In AI, evaluation often focuses on comparisons and trade-offs rather than a single metric.
  • Mentions concepts like:
    • relevancy/faithfulness/groundedness
    • context window limits (how much text can be processed)
    • latency vs accuracy (and cost based on tokens/compute)

6) LLM vs SLM and deployment trade-offs

  • LLM (Large Language Model):
    • typically hosted via API; requires heavy infrastructure.
  • SLM (Small Language Model):
    • can often be run with less compute (sometimes self-hosted or on-device).

Trade-offs:

  • SLM may be cheaper and more private/offline, but may have limitations in reasoning/quality.
  • LLM may be higher quality but can depend on external servers (latency/cost).

7) Practical client requirements: cost, time, and reduced human error

He describes typical enterprise/startup AI demands:

  • automating workflows (tickets, Excel/data updates, customer support, inventory insights)
  • reducing manual effort, cost, and human errors

In consulting-like settings, impact is measured with business metrics such as:

  • time saved, cost reduction, quality improvements, use cases delivered

He notes startups may require fast delivery (“no room for creativity”), unlike larger enterprises with more resources.

8) Servers, data centers, and the scaling problem

  • “Servers” are infrastructure needed for training/inference, usually via cloud providers (AWS/GCP/Azure).
  • AI usage growth increases data-center demand and energy/heat impacts.
  • Mentions longer-term possibilities (e.g., alternative hosting locations, even Mars—mostly speculative), but notes latency challenges and practicality.

9) Responsible AI usage

He closes with the message that using AI responsibly matters:

  • abuse can occur inside organizations
  • AI usage patterns and prompting strategies (“pro/prompt discipline”) may need adjustment to avoid misuse or over-reliance.

Presenters / contributors

  • Ram (host / interviewer)
  • Ishan/Ishaan Katoch (guest; AI Engineer, ex-Samsung R&D)

Original video