Smoke Testing vs Regression Testing: What's the Difference?

Most teams run their entire Playwright suite after every deploy and call it regression testing. When something breaks in production at 11pm, a 45-minute suite isn't useful: you need 10 tests that confirm login works, the API responds, and the core user action completes. This article covers the difference in purpose between smoke and regression testing, how to tag and structure both in a single Playwright suite, and when to run each.

Smoke testing

A smoke test answers one question: is the system fundamentally working?

The name comes from hardware testing — power on a device, see if it smokes. Pass that check and it's worth doing more. Fail, and deeper testing is pointless.

In software, a smoke test is a small, fast subset of tests that verify the most critical paths are operational. Usually 10–20 tests, runnable in under 5 minutes.

• Application starts and loads
• Authentication works
• The most critical user action completes (place an order, submit a form, make a payment)
• Core API endpoints respond

• Edge cases
• Error states
• Secondary features
• Performance under load

When to run smoke tests

• After every deployment (even to production)
• After infrastructure changes (server restarts, config changes)
• As the first step in CI, before the full suite runs
• When you need a quick sanity check before a demo

Smoke test example in Playwright

// smoke.spec.ts
import { test, expect } from '@playwright/test';

test.describe('@smoke', () => {
  test('app loads', async ({ page }) => {
    await page.goto('/');
    await expect(page).toHaveTitle(/MyApp/);
    await expect(page.getByRole('navigation')).toBeVisible();
  });

  test('login works', async ({ page }) => {
    await page.goto('/login');
    await page.getByLabel('Email').fill(process.env.TEST_USER_EMAIL!);
    await page.getByLabel('Password').fill(process.env.TEST_USER_PASSWORD!);
    await page.getByRole('button', { name: 'Log in' }).click();
    await expect(page).toHaveURL('/dashboard');
  });

  test('core API is responding', async ({ request }) => {
    const response = await request.get('/api/health');
    expect(response.status()).toBe(200);
    const body = await response.json();
    expect(body.status).toBe('ok');
  });

  test('order placement works end to end', async ({ page }) => {
    // abbreviated critical path
    await page.goto('/products');
    await page.getByRole('link', { name: 'Laptop Pro' }).click();
    await page.getByRole('button', { name: 'Add to cart' }).click();
    await page.goto('/checkout');
    // ... minimal checkout steps
    await expect(page.getByText('Order confirmed')).toBeVisible();
  });
});

Run smoke tests only:

npx playwright test --grep @smoke

Regression testing

A regression test suite verifies that existing functionality still works after a change. The concern isn't "does the system work at all" — it's "did we break something that was working before."

Regression tests are broader and slower than smoke tests. A complete regression suite might cover hundreds of scenarios and take 20–60 minutes to run.

• All features, including secondary and edge case scenarios
• Error handling
• Boundary conditions
• Cross-browser behavior
• Integration points between services

When to run regression tests

• Before every release
• After significant code changes (new features, refactoring)
• On the main branch after PR merges
• Nightly for large suites

Types of regression testing

Key differences

| | Smoke Testing | Regression Testing |

|---|---|---|

| Purpose | Is the system alive? | Did we break something? |

| Scope | Critical paths only | All features |

| Speed | Fast (2–5 min) | Slow (20–60+ min) |

| Frequency | Every deployment | Before releases, after changes |

| Test count | 10–30 tests | 100s–1000s |

| Failure action | Roll back or stop | Fix before release |

Building the right split

Most teams run their entire suite as regression and have no dedicated smoke layer. This creates a problem: when you deploy to production and something is wrong, you can't quickly verify the critical paths because your 45-minute full regression suite isn't the right tool.

A better structure:

@smoke (run every deploy, 5 min max)
├── App loads
├── Auth works
├── Core feature completes

@regression (run before release, 30-60 min)
├── @smoke tests (included)
├── All other feature tests
├── Edge cases
└── Cross-browser

Tag your most critical tests with @smoke. Run them after every deploy. Run the full suite before every release.

What's not regression testing

The confusion comes from overloading the word "regression" — in common usage, any automated test that runs after a change is called a regression test. In formal usage, regression testing specifically means verifying that previously working functionality still works.

Both usages are fine in practice. What matters is being clear about what you're verifying and why.