前端性能测试神器 – Phantomas

GitHub Logo

PhantomJS-based modular web performance metrics collector. And why phantomas? Well, because 🙂

NPM version
Build Status

Download stats

Requirements

Installation

1
npm install --global phantomas

This will install the latest version of PhantomJS and add a symlink called

1
phantomas

(pointing to

1
./bin/phantomas.js

) to your system’s

1
PATH

You may need to install libfontconfig by running

1
sudo apt-get install libfontconfig1

.

Libraries

phantomas is written in JavaScript, but you can experience it in different languages as well 😉

Python

Latest Version
Supported Python versions

1
pip install phantomas

Features

  • modular approach – each metric is generated by a separate “module”
  • phantomas “core” acts as an events emitter that each module can hook into
  • in-depth metrics such as: number of events bound via jQuery, calls to
    1
    window.write

    or complex and duplicated CSS selectors (via analyze-css)

  • JSON and CSV as available output formats for easy integration with automated reporting / monitoring tools
  • easy integration with Continuous Integration tools via TAP format and assertions handling
  • metrics can be sent via StatsD or stored in elasticsearch
  • easy integration with other nodejs projects via CommonJS module (see API docs)
  • metrics can be emitted from JavaScript code of the page phantomas is run against (thanks to helper functions available in window.__phantomas)
  • device profiles allow phantomas to emulate mobile or tablet (by setting a proper user agent and viewport)
  • ability to run phantomas using WebKit (PhantomJS) or Gecko (SlimerJS) engine (experimental)

Contributors

All the contributors

Usage

phantomas comes as both command line tool and CommonJS module (see API docs) that you can use in your nodejs projects.

Single run

1
2
bash
phantomas https://github.com/macbre/phantomas --verbose

You can measure the performance of your site without requests to 3rd party domains (but allowing CDN that serves your static assets):

1
2
bash
phantomas https://github.com/macbre/phantomas --verbose --no-externals --allow-domain .fastly.net

Parameters

  • 1
    --reporter=[json|csv|tap|plain|statsd|elasticsearch]

    results reporter aka format (

    1
    plain

    is the default one)

  • 1
    --timeout=[seconds]

    timeout for phantomas run (defaults to 15 seconds)

  • 1
    --viewport=[width]x[height]

    phantomJS viewport dimensions (1280×1024 is the default)

  • 1
    --verbose

    writes debug messages to the console

  • 1
    --debug

    run PhantomJS in debug mode

  • 1
    --engine

    select engine used to run the phantomas

    1
    [webkit|gecko]

    experimental

  • 1
    --colors

    forces ANSI colors even when output is piped (eg. via

    1
    less -r

    )

  • 1
    --silent

    don’t write anything to the console

  • 1
    --progress

    shows page loading progress bar (disables verbose mode)

  • 1
    --log=[log file]

    log to a given file

  • 1
    --modules=[moduleOne],[moduleTwo]

    run only selected modules

  • 1
    --include-dirs=[dirOne],[dirTwo]

    load modules from specified directories

  • 1
    --skip-modules=[moduleOne],[moduleTwo]

    skip selected modules

  • 1
    --user-agent='Custom user agent'

    provide a custom user agent (will default to something similar to

    1
    phantomas/0.6.0 (PhantomJS/1.9.0; linux 64bit)

    )

  • 1
    --config=[JSON/YAML config file]

    uses JSON or YAML-formatted config file to set parameters

  • 1
    --cookie='bar=foo;domain=url'

    document.cookie formatted string for setting a single cookie

  • 1
    --cookies-file=[JAR file]

    specifies the file name to store the persistent Cookies

  • 1
    --no-externals

    block requests to 3rd party domains

  • 1
    --allow-domain=[domain],[domain]

    allow requests to given domain(s) – aka whitelist

  • 1
    --block-domain=[domain],[domain]

    disallow requests to given domain(s) – aka blacklist

  • 1
    --auth-user

    sets the user name used for HTTP authentication

  • 1
    --auth-pass

    sets the password used for HTTP authentication

  • 1
    --disable-js

    disable JavaScript on the page that will be loaded

  • 1
    --analyze-css

    emit in-depth CSS metrics experimental

  • 1
    --film-strip

    register film strip when page is loading experimental

  • 1
    --film-strip-dir=[dir path]

    folder path to output film strip (default is

    1
    ./filmstrip

    directory)

  • 1
    --film-strip-prefix

    film strip files name prefix (defaults to

    1
    screenshot

    )

  • 1
    --page-source

    save page source to file experimental

  • 1
    --page-source-dir=[dir path]

    folder path to output page source (default is

    1
    ./html

    directory) experimental

  • 1
    --assert-[metric-name]=value

    assert that given metric should be less or equal the value

  • 1
    --screenshot=[file name]

    render fully loaded page to a given file

  • 1
    --har=[file name]

    save HAR to a given file

  • 1
    --wait-for-event=[phantomas event name]

    wait for a given phantomas event before generating a report, timeout setting still applies (e.g.

    1
    --wait-for-event "done"

    )

  • 1
    --wait-for-selector=[CSS selector]

    wait for an element matching given CSS selector before generating a report, timeout setting still applies (e.g.

    1
    --wait-for-selector "body.loaded"

    )

  • 1
    --stop-at-onload

    stop phantomas immediately after

    1
    onload

    event

  • 1
    --scroll

    scroll down the page when it”s loaded

  • 1
    --post-load-delay=[seconds]

    wait X seconds before generating a report, timeout setting still applies

  • 1
    --ignore-ssl-errors

    ignores SSL errors, such as expired or self-signed certificate errors

  • 1
    --ssl-protocol

    sets the SSL protocol for secure connections

    1
    [sslv3|sslv2|tlsv1|any]
  • 1
    --proxy=[host:port]

    specifies the proxy server to use

  • 1
    --proxy-auth=[username:password]

    specifies the authentication information for the proxy

  • 1
    --proxy-type=[http|socks5|none]

    specifies the type of the proxy server (default is http)

  • 1
    --phone

    force viewport and user agent of a mobile phone

  • 1
    --tablet

    force viewport and user agent of a tablet

  • 1
    --spy-eval

    report calls to eval()

Multiple runs

Simply provide

1
--runs

option:

1
2
bash
phantomas https://github.com/macbre/phantomas --verbose --runs 5

Only

1
plain

(the default one) and

1
json

reporters are currently supported in multiple runs mode.

Metrics

Current number of metrics: 134

Units:

  • ms for time
  • bytes for size

Requests monitor (core module)

Due to PhantomJS issue #10156 body size related metrics are not reliable

  • requests: total number of HTTP requests made
  • gzipRequests: number of gzipped HTTP responses
  • postRequests: number of POST requests
  • httpsRequests: number of HTTPS requests
  • notFound: number of HTTP 404 responses
  • timeToFirstByte: time it took to receive the first byte of the first response (that was not a redirect)
  • timeToLastByte: time it took to receive the last byte of the first response (that was not a redirect)
  • bodySize: size of the content of all responses
  • contentLength: size of the content of all responses (based on
    1
    Content-Length

    header)

  • httpTrafficCompleted: time it took to receive the last byte of the last HTTP response

AJAX requests

  • ajaxRequests: number of AJAX requests

Assets types

Due to PhantomJS issue #10156 body size related metrics are not reliable

  • htmlCount: number of HTML responses
  • htmlSize: size of HTML responses
  • cssCount: number of CSS responses
  • cssSize: size of CSS responses
  • jsCount: number of JS responses
  • jsSize: size of JS responses
  • jsonCount: number of JSON responses
  • jsonSize: size of JSON responses
  • imageCount: number of image responses
  • imageSize: size of image responses
  • webfontCount: number of web font responses
  • webfontSize: size of web font responses
  • videoCount: number of video responses
  • videoSize: size of video responses
  • base64Count: number of base64 encoded “responses” (no HTTP request was actually made)
  • base64Size: size of base64 encoded “responses”
  • otherCount: number of other responses
  • otherSize: size of other responses

Cache Hits

Metrics are calculated based on

1
Age

and

1
X-Cache

headers added by Varnish / Squid servers

  • cacheHits: number of cache hits
  • cacheMisses: number of cache misses
  • cachePasses: number of cache passes

Headers

  • headersCount: number of requests and responses headers
  • headersSentCount: number of headers sent in requests
  • headersRecvCount: number of headers received in responses
  • headersSize: size of all headers
  • headersSentSize: size of sent headers
  • headersRecvSize: size of received headers
  • headersBiggerThanContent: number of responses with headers part bigger than the response body

Domains

  • domains: number of domains used to fetch the page
  • maxRequestsPerDomain: maximum number of requests fetched from a single domain
  • medianRequestsPerDomain: median of requests fetched from each domain

Cookies

  • cookiesSent: length of cookies sent in HTTP requests
  • cookiesRecv: length of cookies received in HTTP responses
  • domainsWithCookies: number of domains with cookies set
  • documentCookiesLength: length of
    1
    document.cookie
  • documentCookiesCount: number of cookies in
    1
    document.cookie

DOM complexity

Metrics listed below are generated after the full page load

  • globalVariables: number of JS globals variables
  • globalVariablesFalsy: number of JS global variables that cast to false
  • bodyHTMLSize: the size of body tag content (
    1
    document.body.innerHTML.length

    )

  • commentsSize: the size of HTML comments on the page
  • hiddenContentSize: the size of content of hidden elements on the page (with CSS
    1
    display: none

    )

  • whiteSpacesSize: the size of text nodes with whitespaces only
  • DOMelementsCount: total number of HTML element nodes
  • DOMelementMaxDepth: maximum level on nesting of HTML element node
  • DOMidDuplicated: number of duplicated IDs found in DOM
  • iframesCount: number of iframe nodes
  • nodesWithInlineCSS: number of nodes with inline CSS styling (with
    1
    style

    attribute)

  • imagesScaledDown: number of nodes that have images scaled down in HTML
  • imagesWithoutDimensions: number of
    1
    <img />

    nodes without both

    1
    width

    and

    1
    height

    attribute

DOM queries

  • DOMqueries: the sum of all four metrics below
  • DOMqueriesWithoutResults: number of DOM queries that returned nothing
  • DOMqueriesById: number of
    1
    document.getElementById

    calls

  • DOMqueriesByClassName: number of
    1
    document.getElementsByClassName

    calls

  • DOMqueriesByTagName: number of
    1
    document.getElementsByTagName

    calls

  • DOMqueriesByQuerySelectorAll: number of
    1
    document.querySelectorAll

    calls

  • DOMinserts: number of DOM nodes inserts
  • DOMqueriesDuplicated: number of DOM queries called more than once
  • DOMqueriesAvoidable: number of repeated uses of a duplicated query

DOM mutations

These metrics are only available when running phantomas using Gecko engine (

1
--engine=gecko

)

  • DOMmutationsInserts: number of node inserts
  • DOMmutationsRemoves: number of node removes
  • DOMmutationsAttributes: number of DOM nodes attributes changes

Event listeners

  • eventsBound: number of
    1
    EventTarget.addEventListener

    calls

  • eventsDispatched: number of
    1
    EventTarget.dispatchEvent

    calls

  • eventsScrollBound: number of
    1
    scroll

    event bounds to

    1
    window

    or

    1
    document

Window performance

Times below are relative to

1
responseEnd

entry in NavigationTiming (represented by

1
timeToLastByte

metric). See NavigationTiming spec for more information.

  • domInteractive: time it took to parse the HTML and construct the DOM
  • domContentLoaded: time it took to construct both DOM and CSSOM, no stylesheets are blocking JavaScript execution (i.e. onDOMReady)
  • domContentLoadedEnd: time it took to finish handling of onDOMReady event experimental
  • domComplete: time it took to load all page resources, the loading spinner has stopped spinning
  • timeBackend: time to the first byte compared to the total loading time (in %)
  • timeFrontend: time to window on load compared to the total loading time (in %)

Repaints

These metrics are only available when running phantomas using Gecko engine (

1
--engine=gecko

)

  • repaints: number of repaints of the current document
  • firstPaint: time it took to perform the first paint (time is relative to
    1
    responseEnd

    event)

Requests statistics

Time is total duration, from the start of the request to the receipt of the final byte in the response. Latency is the time to load the first byte in the response.
https://developers.google.com/chrome-developer-tools/docs/network

Includes

1
HTTP 200

responses only

  • smallestResponse: the size of the smallest response
  • biggestResponse: the size of the biggest response
  • fastestResponse: the time to the last byte of the fastest response
  • slowestResponse: the time to the last byte of the slowest response
  • smallestLatency: the time to the first byte of the fastest response
  • biggestLatency: the time to the first byte of the slowest response
  • medianResponse: median value of time to the last byte for all responses
  • medianLatency: median value of time to the first byte for all responses

Requests to

  • requestsToFirstPaint: number of HTTP requests it took to make the first paint
  • domainsToFirstPaint: number of domains used to make the first paint
  • requestsToDomContentLoaded: number of HTTP requests it took to make the page reach
    1
    DomContentLoaded

    state

  • domainsToDomContentLoaded: number of domains used to make the page reach DomContentLoaded state
  • requestsToDomComplete: number of HTTP requests it took to make the page reach
    1
    DomComplete

    state

  • domainsToDomComplete: number of domains used to make the page reach DomComplete state

keepAlive

Monitors the use of

1
Connection: close

and

1
Keep-Alive
  • closedConnections: number of requests not keeping the connection alive and slowing down the next request

localStorage

  • localStorageEntries: number of entries in local storage

jQuery

Requires jQuery 1.8.0+

  • jQueryVersion: version of jQuery framework (if loaded)
  • jQueryVersionsLoaded: number of loaded jQuery “instances” (even in the same version)
  • jQueryOnDOMReadyFunctions: number of functions bound to
    1
    onDOMReady

    event

  • jQueryWindowOnLoadFunctions: number of functions bound to
    1
    windowOnLoad

    event

  • jQuerySizzleCalls: number of calls to Sizzle (including those that will be resolved using
    1
    querySelectorAll

    )

  • jQueryEventTriggers: number of jQuery event triggers
  • jQueryDOMReads: number of DOM read operations
  • jQueryDOMWrites: number of DOM write operations
  • jQueryDOMWriteReadSwitches: number of read operations that follow a series of write operations (will cause repaint and can cause reflow)

Static assets

  • assetsNotGzipped: static assets that were not gzipped
  • assetsWithQueryString: static assets requested with query string (e.g. ?foo) in URL
  • assetsWithCookies: number of static assets requested from domains with cookie set
  • smallImages: images smaller than 2 KiB that can be base64 encoded
  • smallCssFiles: number of CSS assets smaller than 2 KiB that can be inlined or merged
  • smallJsFiles: number of JS assets smaller than 2 KiB that can be inlined or merged
  • multipleRequests: number of static assets that are requested more than once

Caching

  • cachingNotSpecified: responses with no caching header sent (either
    1
    Cache-Control

    or

    1
    Expires

    )

  • cachingTooShort: responses with too short (less than a week) caching time
  • cachingDisabled: responses with caching disabled (
    1
    max-age=0

    )

  • oldCachingHeaders: responses with old, HTTP 1.0 caching headers (
    1
    Expires

    and

    1
    Pragma

    )

Time to first asset

  • timeToFirstCss: time it took to receive the last byte of the first CSS
  • timeToFirstJs: time it took to receive the last byte of the first JS
  • timeToFirstImage: time it took to receive the last byte of the first image

Redirects

  • redirects: number of HTTP redirects (either 301, 302 or 303)
  • redirectsTime: time it took to send and receive redirects

JavaScript bottlenecks

  • documentWriteCalls: number of calls to either
    1
    document.write

    or

    1
    document.writeln
  • evalCalls: number of calls to
    1
    eval

    (either direct or via

    1
    setTimeout

    /

    1
    setInterval

    )

JavaScript errors

Error message and backtrace will be emitted as offenders

  • jsErrors: number of JavaScript errors

JavaScript console and alert

  • windowAlerts: number of calls to
    1
    alert
  • windowConfirms: number of calls to
    1
    confirm
  • windowPrompts: number of calls to
    1
    prompt
  • consoleMessages: number of calls to
    1
    console.*

    functions

Main request

Analyzes bits of data pertaining to the main request only

  • statusCodesTrail: comma-separated list of HTTP status codes that main request followed through (could contain a single element if the main request is a terminal one)

Document height

Lazy-loadable images

  • lazyLoadableImagesBelowTheFold: number of images displayed below the fold that can be lazy-loaded

Optional metrics

The following metrics are emitted only when certain options are passed to phantomas

  • blockedRequests: number of requests blocked due to domain filtering (emitted only when in
    1
    --no-externals

    /

    1
    --block-domain

    mode)

CSS metrics

This is an experimental feature. Use

1
--analyze-css

option to enable it.

Take a look at analyze-css README for the full list of metrics.

  • cssParsingErrors: number of CSS files (or embeded CSS) that failed to be parse by analyze-css

Reporters

phantomas provides a number of reporters that can format the run results and send them to various tools. Use

1
--reporter

(or

1
-R

shortcut) option to use one.

Formatters

Results can be emitted as TAP, CSV and JSON.

1
plain

format is most useful for human beings 🙂

Parameters

Formatters can be provided with colon separated list of options:

1
$ phantomas http://foo.net -R csv:no-header:timestamp

This will omit CSV headers row and add current timestamp as the first column, so you can append the results line to a growing file.

CSV
  • 1
    no-header

    – omit CSV header

  • 1
    timestamp

    – add the current timestamp as the first column

  • 1
    url

    – add the URL as the first column

Elasticsearch
  • 1
    <host>:<port>:<index>:<type>

    – shorthand for

    1
    --elasticsearch-*

    options

JSON
  • 1
    pretty

    – emits pretty printed JSON

Plain
  • 1
    no-color

    – disable ANSI colors

StatsD
  • 1
    <host>:<port>:<prefix>

    – shorthand for

    1
    --statsd-host

    ,

    1
    --statsd-port

    and

    1
    --statsd-prefix

    (you don’t need to provide all three options)

TAP
  • 1
    no-skip

    – don’t print out metrics that were skipped

StatsD integration

Metrics from phantomas run can be sent directly to StatsD and then graphed using graphite, graphene or any other tool of your choice. For instance:

1
$ phantomas http://app.net/start -R statsd --statsd-host stats.app.net --statsd-port 8125 --statsd-prefix 'myApp.mainPage.'

or

1
$ phantomas http://app.net/start -R statsd:stats.app.net:8125:myApp.mainPage.

will sent metrics to StatsD running on

1
stats.app.net:8125

and prefix them with ‘myApp.mainPage’.

Save metrics to Elasticsearch

Metrics from phantomas run can be outputted directly in Elasticsearch :

Parameters
  • 1
    --elasticsearch-host=[ip]

    Elasticsearch instance ip (default : 127.0.0.1)

  • 1
    --elasticsearch-port=[port]

    Elasticsearch instance port (default : 9200)

  • 1
    --elasticsearch-index=[index_name]

    Name of the index to use

  • 1
    --elasticsearch-type=[type_name]

    Name of the document type to use

Or by using reporter options (

1
<host>:<port>:<index>:<type>

):

1
$ phantomas http://app.net/start -R elasticsearch:es.app.net::app:phantomas_metrics

Note: as

1
<port>

option was skipped a default value will be used (

1
9200

).

Engines

phantomas can be run using PhantomJS (WebKit-powered headless browser) or SlimerJS (Gecko-based non headless browser, run using xfvb). Use either

1
--engine=[webkit|gecko]

or

1
--webkit

/

1
--gecko

parameters to choose one. Please note that support for SlimerJS is experimental at this point.

PhantomJS

All required binaries are installed by npm. No extra work needed here 🙂

SlimerJS

In order to use SlimerJS install the following Debian/Ubuntu packages:

1
sudo aptitude install xvfb libasound2 libgtk2.0-0

For developers

Custom modules

You can load your own, custom phantomas modules using

1
--include-dirs

option:

1
phantomas --include-dirs /my/path/to/custom/modules/ --url http://example.com
1
/my/path/to/custom/modules/

directory should contain custom modules, each in its own directory, e.g.

1
/my/path/to/custom/modules/fooBar/fooBar.js

.

Let’s make Web a bit faster!

Slides

Blogosphere

Introductions to phantomas and use cases:

Videos

Utilities

Use grunt to automate daily dev tasks, including your’s application web performance, via these great tools: