Jarom Hulet, a data science leader at Toyota Financial Services, provides insights into the practical application of data science in the corporate world. In a recent discussion, he outlined his views on the strategic use of generative AI, the importance of foundational skills for career changers, and the balance between rigorous experimentation and business objectives.
Key Takeaways
- Generative AI significantly enhances productivity in coding and research but poses a risk of creating superficial knowledge if over-relied upon.
- Well-designed experiments are the gold standard for causal analysis, but their implementation must be economically viable for the business.
- Aspiring data scientists should focus on a deep mastery of fundamental concepts, as this is a key differentiator in a competitive job market.
- Hulet's approach to writing and professional development is driven by a combination of solving immediate work-related problems and pursuing personal curiosity.
The Role of Experimentation in Business
In the field of data science, experimental data is often considered the most reliable source for causal analysis. However, Jarom Hulet emphasizes that in a business context, the pursuit of statistical perfection must be weighed against economic reality. He notes that while observational data is more abundant and cheaper to acquire, it should not completely replace controlled tests.
Hulet advocates for what his mentor calls a pragmatic philosophy: "some testing is better than no testing." This approach acknowledges that the goal of experimentation in industry is not simply to learn, but to generate value. Therefore, the cost of an experiment must be justified by its potential economic benefit to the organization.
Defining a Viable Experiment
According to Hulet, a "minimal viable experiment" is one that key stakeholders approve and is projected to have a positive financial impact. This requires data scientists to understand business constraints and clearly communicate how these limitations might affect the insights gained from the experiment.
He believes many data scientists lack an "experimental state of mind," a term coined by Paul Rosenbaum. This mindset involves actively seeking opportunities to test hypotheses rather than relying solely on available observational data. By balancing statistical rigor with business needs, data scientists can design experiments that provide valuable, actionable insights without incurring excessive costs.
Generative AI: A Double-Edged Sword for Productivity
Generative AI has become an indispensable tool in Hulet's daily workflow, significantly boosting his productivity. He primarily uses it for two key tasks: coding and research. AI models can quickly generate basic Python code snippets, freeing up his time to focus on more complex problems. He notes that the quality of AI-generated code has improved substantially, requiring less debugging than in the past.
"I often find myself telling ChatGPT to write a somewhat simple function, and I respond to a message or read an email while it writes the code," Hulet explained.
AI also accelerates the debugging process. Instead of manually searching through forums like Stack Overflow, he can paste error messages directly into an AI tool to get immediate suggestions. In research, he uses generative AI as a study companion to clarify confusing concepts from academic papers or textbooks and to discover relevant new resources for specific problems.
The Risk of Artificial Intelligence
Despite its benefits, Hulet expresses concern that generative AI could be used as a substitute for, rather than a supplement to, human intelligence. He draws a parallel to Socrates' skepticism of writing, fearing it would weaken memory. The constant availability of AI could discourage deep learning and memorization, leading to a superficial understanding of complex topics.
"I’m concerned that we will use artificial intelligence as a substitute for actual intelligence rather than a supplement and multiplier," he stated. To counter this, he uses AI as a secondary resource to support his primary studies from books and papers, ensuring he develops a robust understanding rather than just gathering quick answers.
Guidance for Aspiring Data Scientists
For individuals looking to transition into a data science career, Hulet offers updated advice tailored to the current landscape. While many tactics for breaking into the field remain relevant, he highlights two crucial new considerations.
First, job applicants should recognize that not all data science roles are focused on generative AI. He advises candidates to tailor their resumes to match the specific job description, distinguishing between "traditional" data science positions and those centered on AI. Sending a resume saturated with GenAI experience to a traditional role, or vice versa, can be ineffective.
Second, and most importantly, is the need for an "intellectual mastery of the basics." In an era where AI can provide instant answers, deep, genuine understanding is a powerful differentiator. Hulet has observed that many candidates from accelerated master's programs possess only a superficial knowledge of core concepts.
Common Interview Pitfalls
Hulet shared an example from interviews where candidates mistakenly identify "accuracy" as a performance metric for regression models. This error reveals a shallow understanding, as accuracy is typically used for classification problems. Such mistakes quickly expose candidates who lack foundational knowledge.
He urges aspiring professionals to develop a deep comprehension of fundamental principles to engage in meaningful technical conversations during interviews and, ultimately, to solve complex problems effectively on the job.
The Sources of Inspiration and Knowledge
Hulet's writing and continuous learning are fueled by what he describes as a combination of necessity and curiosity. Many of his article topics originate from practical challenges he faces at work. When a project requires a deeper understanding of a subject, such as linear programming, he immerses himself in it by reading textbooks, replicating processes in Python, and then sharing his newfound expertise through writing.
His curiosity also drives him to explore diverse fields like philosophy. He finds that taking time for reflection—what he calls "solitude"—is essential for internalizing information and generating unique insights. This process of reading, researching, and then deliberately thinking about the material allows him to connect concepts from different domains to his work in data science.
When it comes to crafting articles, he uses visuals and code snippets to clarify complex ideas that would be difficult to explain with text alone. This approach ensures his content is not only informative but also accessible and intuitive for readers, helping them grasp what's happening "under the hood" of complex algorithms and processes.