Interviewing for Expertise You Do Not Have

Before you design the interview, decide which job owns the mandate:

• PlatformCreate the data and technical foundation that other AI work requires.
• AdoptionChange how employees perform recurring work.
• GovernanceEstablish the rules, controls, and accountability for AI use.
• BuildDeliver AI systems, automations, or product capabilities.

A strong leader may contribute to all four. One of them must still be the main job.

Then complete this sentence:

The answer must describe a business result, not a list of activities.

If you cannot complete the sentence, stop work on the interview. The company has not defined the role well enough to evaluate a candidate for it.

No interview process can correct an undefined mandate.

The memorized technical question. Someone gives you a “good question to ask AI candidates.” You ask it. The candidate gives a detailed answer.

You still do not know if the answer is correct.

A technical question is useful only when someone qualified can assess the answer and the question reflects the actual work.

The borrowed expert. You ask a technical friend, an investor’s CTO, or a board member’s contact to conduct an interview.

This can work if you define the assessment first.

Without clear instructions, the expert will interview for the job they know. An engineering leader will test engineering depth. A data scientist will test model knowledge.

Give the expert the written mandate. State what the candidate must be able to do. Ask the expert to test those capabilities, not general technical intelligence.

Credential weight. Certifications, titles, and brand-name employers are useful inputs. They are not sufficient evidence.

A successful project at a large company may have depended on a team, data foundation, and budget that your company does not have. A certification can show structured study. It cannot prove that a candidate can move a system from demonstration to daily use.

Use credentials to decide what to verify. Do not use them as a substitute for verification.

You need a reliable way to verify technical depth. The method must match the main job.

A qualified evaluator can review architecture, code, data practices, security controls, or production operations as required.

Your part of the interview has a different purpose.

You must determine whether the candidate can make good decisions inside your business.

Companies selected the wrong problem. They optimized the wrong measure. Leaders and technical teams did not share the same objective. Priorities changed before teams could produce results. Systems did not fit the workflows they were supposed to improve. Read the RAND report.

A technically capable hire can still:

• Build something that employees do not use.
• Automate a task that does not affect a business result.
• Select a complex solution when a simple one would work.
• Ignore an approval, security, or adoption constraint.
• Spend the first year negotiating a mandate that leadership never defined.

These are failures of judgment, mandate, and fit.

You are qualified to evaluate judgment and fit. The mandate is yours to define.

The interview problem is not how to become qualified to grade every AI answer. It is:

In July 2026, Coinbase reported that the share of merged code generated by AI had reached roughly 100%.

Coinbase rebuilt its engineering interviews around three dimensions of AI fluency:

Usage

Can the candidate select and use the right tool to produce a better result?

Application

Does the candidate know when AI is and is not the right solution?

Understanding limits

Does the candidate understand failure, privacy, security, and the need for human judgment?

Coinbase applies this definition to junior and senior engineering candidates, and states that its interview signals ultimately test judgment and taste. Read the Coinbase account.

Your role may not be an engineering role, and Coinbase is not a model for every company. But its framework makes an important distinction.

Using AI is only one part of AI fluency.

A strong candidate must also decide:

• Which problems are suitable for AI.
• What evidence would justify an investment.
• Where the system can fail.
• Which risks are acceptable.
• Which decisions must remain with people.
• When the company should stop.

These are business decisions with technical inputs. You already evaluate this type of judgment when you hire leaders in other fields.

Ask:

Do not score the answer because it contains a failure. Failure stories can be prepared as easily as success stories.

Test whether the account holds together:

• What did you believe at the start?
• Which evidence showed that you were wrong?
• Who identified the problem first?
• What did the failure cost?
• What changed afterward?
• Is that change still in place?
• Who else can verify the account?

This is not a technical evaluation. It is interviewing for scar tissue, which executives have done their whole careers.

Ask the candidate to explain an important past decision. If the answer depends on jargon, say:

This role must explain uncertainty, cost, and risk to employees, executives, and possibly the board. The interview shows whether the candidate can do that work.

The strongest assessment available to you begins with a real workflow.

Select one recurring process from your company. It could be a monthly report, a client brief, or a support escalation.

Describe the process as it actually operates. Include its delays, exceptions, manual steps, and informal workarounds. Do not make the description cleaner than the work.

Then say:

Let the candidate run the next part of the interview.

Do not score the candidate only on the questions asked. Score the sequence of the questions and how the candidate uses each answer.

Listen for questions about:

• The result the process must produce.
• How the company measures that result now.
• Where the required data lives, and who can approve access.
• Which decisions require human approval.
• What the people who perform the work believe.
• What the company has already tried.

Do not provide all the information at once. Let the candidate find what is missing.

After the discovery period, ask for four outputs:

1. What do you think the problem might be?
2. What important facts are still unknown?
3. What would you measure before you change the process?
4. What would you do first, and what would you not do yet?

You can now observe how the candidate operates in an uncertain business situation.

Does the candidate separate facts from assumptions? Find the business constraint? Narrow the first step? Protect an important human decision? Select a reversible test? Name a reason to stop?

The candidate who proposes a solution in the first ten minutes is showing you how they will operate in month one: building before mapping, on your payroll.

The candidate who performs discovery but never makes a decision is also showing you how they will operate.

You want disciplined movement: neither immediate construction nor permanent discovery.

Technical capability still requires separate verification. Use a combination of:

• A structured interview with an evaluator who has the mandate and knows what capability to test.
• A review of past work, with clear questions about the candidate’s contribution and what happened after launch.
• A small, job-relevant work simulation that tests framing, tradeoffs, and communication. Pay for substantial work when appropriate.
• Specific references that verify what the candidate owned, what reached production, who used it, and whether it remains in use.

Do not ask for a general verdict that the candidate is “technical enough.” Define the required evidence before the assessment begins.

The candidates you most want may have seen AI roles with no real mandate.

They have seen companies hire a person instead of making a decision. They have seen leaders ask for transformation without granting access, authority, budget, or time.

They will look for evidence that your role is different:

• A written mandate.
• A clear main job.
• A named executive sponsor.
• A defined path for data access.
• A realistic first ninety days.
• A clear method for measuring progress.

Strong candidates take prepared jobs. The preparation that makes them easier to evaluate is the same preparation that attracts them.