Spark Monitoring

Posted on | Edited on | In | Views:

Web UI

  • WebUI, WebUITab

    SparkUI, SparkUITab
    WorkerWebUI, MasterWebUI(Standalone)
    HistoryServer(Yarn or Mesos)

Restful

  • StandaloneRestServer(RestSubmissionServer)

  • StandaloneSubmitRequestServlet, StandaloneKillRequestServlet, StandaloneStatusRequestServlet

Metrics

  • MetricsSystem, MetricsConfig

  • Source(MasterSource, ApplicationSource, WorkerSource, ExecutorSource, DAGSchedulerSource, ShuffleMetricsSource, BlockManagerSource, StreamingSource, JvmSource)

  • Sink(ConsoleSink, CsvSink, GraphiteSink, JmxSink, MetricsServlet, Slf4jSink, StatsdSink)