Thursday, 26 March 2020

Linux Metricbeat Counters Mapping

Performance Counters Linux

We are using counters provided by metricbeat modules, iostat, sar

Server Stats

CPU [Command: /proc/stat]           

CPU Total

( ( system.cpu.user.pct + system.cpu.system.pct ) / system.cpu.cores ) * 100

Total User

( system.cpu.user.pct / system.cpu.cores ) * 100

Total System

( system.cpu.system.pct / system.cpu.cores ) * 100

CPUIOWait

( system.cpu.iowait.pct / system.cpu.cores ) * 100
Above 4 metrics were extracted from metricsets system.cpu.user.pct, system.cpu.system.pct and system.cpu.cores
system.cpu.user.pct The percentage of CPU time spent in user space. On multi-core systems, you can have percentages  that are greater than 100%. For example, if 3 cores are at 60% use, then the system.cpu.user.pct will be 180%.
system.cpu.system.pct The percentage of CPU time spent in kernel space.
system.cpu.cores The number of CPU cores present on the host.
system.cpu.iowait.pct The percentage of CPU time spent in wait (on disk)
               

Memory [Command: /proc/meminfo]

Total Memory (MB)

 ( system.memory.total / 1024 / 1024 )
Total amount of physical RAM in the machine

Used Memory (MB)

( system.memory.used.bytes / 1024 / 1024 )
Amount of physical memory used by system

Actual Used Memory (MB) { MemFree - Buffers – Cached }

 ( system.memory.actual.used.bytes / 1024 / 1024 )        

Total Swap Memory (MB)

( system.memory.swap.total.bytes / 1024 / 1024 )

Used Swap Memory (MB)

( system.memory.swap.used.bytes / 1024 / 1024 )

Disk [Command: iostat -dxk]

DiskReadsPerSec

The number of read requests that were issued to the device per second.

DiskWritesPerSec

The number of write requests that were issued to the device per second.

DiskReadKBPerSec

The number of read requests that were issued to the device per second.

DiskWriteKBPerSec

The number of kilobytes written to the device per second.

DiskQueueLength

The average queue length of the requests that were issued to the device.

DiskReadWaitTime

The average time (in milliseconds) for read requests issued to the device to be served.

DiskWriteWaitTime

The average time (in milliseconds) for write requests issued to the device to be served.

Network [Command: sar -n DEV]

ReceivedBytesRate

Total number of kilobytes received per second.

TransmittedBytesRate

Total number of kilobytes transmitted per second.

PacketsReceivedRate

Total number of packets received per second.

PacketsTransmittedRate

Total number of packets transmitted per second.

MongoDB Insights

the server information.

MongoDB operation counters [Command: db.serverStatus().opcounters]

This visualization gives count of following number of operations since mongo instance start

Command

The total number of commands issued to the database since the mongod instance last started.

Delete

The total number of delete operations received since the mongod instance last started.

GetMore

The total number of getmore operations received since the mongod instance last started.

Insert

The total number of insert operations received since the mongod instance last started.

Query

The total number of queries received since the mongod instance last started.

Update

The total number of update operations received since the mongod instance last started.

MongoDB WiredTiger Cache [Command: db.serverStatus().wiredTiger.cache]

WiredTiger is tefault srorage engines used in mongodb. It has some cool features like MVCC, Document level concurrency and journal (write-ahead log).
Default internal cache size is larger of either:
·         50% of (RAM - 1GB), or
·         256 MB

Max

Maximum cache size

Used

Size in byte of the data currently in cache.

Dirty

Size in bytes of the dirty data in the cache.
This is the size of the dirty data in the cache. This value should be less than the bytes currently in the cache value. Looking at these values, we can determine if we need to up the size of the cache for our instance. Additionally, we can look at the wiredTiger.cache.bytes read into cache value for read-heavy applications. If this value is consistently high, increasing the cache size may improve overall read performance.

MongoDB asserts [Command: db.serverStatus().asserts]

number of assertions raised since the MongoDB process started. If there are non-zero values for the asserts, you should check log file for information.

Message

The number of message assertions raised since the MongoDB process started.

Regular

The number of regular assertions raised since the MongoDB process started.

Rollover

The number of times that the rollover counters have rolled over since the last time the MongoDB process started. The counters will rollover to zero after 230 assertions. Use this value to provide context to the other values in the asserts data structure.

User

The number of “user asserts” that have occurred since the last time the MongoDB process started. These are errors that user may generate, such as out of disk space or duplicate key. You can prevent these assertions by fixing a problem with your application or deployment.

Warning

the number of warnings raised since the MongoDB process started.

MongoDB Concurrent transactions Read [Command: db.serverStatus().wiredTiger.concurrentTransactions]

Returns information on the number of concurrent of read and write transactions allowed into the WiredTiger storage engine. Default value is 128.

Read Available

Number of concurrent read tickets available.

Read Used

Number of concurrent read transaction in progress.

Write Available

Number of concurrent read tickets available.

Write Used

Number of concurrent read transaction in progress.

Modelling - Mongo DB - Wired Tiger - Log Writes [Command: db.serverStatus().wiredTiger.log]

Log Writes

Number of write operations.

Modelling - Mongo DB - Wired Tiger - Log Writes (Bytes) [Command: db.serverStatus().wiredTiger.log]

Log Writes (Bytes)

Number of bytes written into the log.

Modelling - Mongo DB - Wired Tiger - Log Syncs [Command: db.serverStatus().wiredTiger.log]

Log Syncs

Number of sync operations.

Modelling - Mongo DB - Wired Tiger - Log Max Size (Bytes) [Command: db.serverStatus().wiredTiger.log]

Log Max Size (Bytes)

Maximum file size. This is constant value set.

Modelling - Mongo DB - Wired Tiger - Log Flushes [Command: db.serverStatus().wiredTiger.log]

Log Flushes

Number of flush operations.

Modelling - Mongo DB - Wired Tiger - Log Scans [Command: db.serverStatus().wiredTiger.log]

Log Scans

Number of scan operations.

Modelling - Mongo DB - Wired Tiger - Log Size [Command: db.serverStatus().wiredTiger.log]

Log Size (Bytes)

Total log size in bytes.

Modelling - Mongo DB - Wired Tiger - Log Cache Pages Read [Command: db.serverStatus().wiredTiger.log]

Cache Pages Read

Number of pages read into the cache. If this value is consistently high, increasing the cache size may improve overall read performance.

Modelling - Mongo DB - Wired Tiger - Log Cache Pages Write [Command: db.serverStatus().wiredTiger.log]

Cache Pages Write

Number of pages written from the cache.

Modelling - Mongo DB - Wired Tiger - Cache Pages Evicted[Command: db.serverStatus().wiredTiger.log]

Cache Pages Evicted

Number of pages evicted from the cache.

Modelling - Mongo DB - Wired Tiger - Cache Dirty Bytes[Command: db.serverStatus().wiredTiger.log]

Cache Dirty Bytes

Size in bytes of the dirty data in the cache. This value should be less than the bytes currently in the cache value.

Modelling - Mongo DB - Resident Memory (MB) [Command: db.serverStatus().mem]

Resident Memory (MB)

The amount of RAM, in megabytes (MB), currently used by the database process.

Modelling - Mongo DB - Virtual Memory (MB) [Command: db.serverStatus().mem]

Virtual Memory (MB)

The amount, in megabytes (MB), of virtual memory used by the mongod process.

Modelling - Mongo DB - Memory Stats (MB) [Command: db.serverStatus().mem]

Resident

The amount of RAM, in megabytes (MB), currently used by the database process.

Virtual

The amount, in megabytes (MB), of virtual memory used by the mongod process.

Mapped

The amount of mapped memory, in megabytes (MB), used by the database. Because MongoDB uses memory-mapped files, this value is likely to be to be roughly equivalent to the total size of your database or databases.

Mapped with Journal

The amount of mapped memory, in megabytes (MB), including the memory used for journaling.

Modelling - Mongo DB – Objects

Modelling - Mongo DB – Objects

Modelling - Mongo DB - Total Connections Created [Command: db.serverStatus().connections]

Total Connections Created

The total number of current clients connected to this instance

Modelling - Mongo DB - Current Connections Created [Command: db.serverStatus().connections]

Current Connections Created

The total number of current clients connected to this instance

Modelling - Mongo DB - Available Connections Created [Command: db.serverStatus().connections]

Available Connections Created

The total number of unused connections available to clients for this instance

Modelling - Mongo DB - Indexes and Collections

Indexes

Number of indexes

Collections

Number of collections

MongoDB Journalling

To provide durability in the event of a failure, MongoDB uses write ahead logging to journal files in Disk. WiredTiger uses checkpoints to recover.
With journaling, WiredTiger creates one journal record for each client initiated write operation. The journal record includes any internal write operations caused by the initial write. For example, an update to a document in a collection may result in modifications to the indexes; WiredTiger creates a single journal record that includes both the update operation and its associated index modifications.
MongoDB configures WiredTiger to use in-memory buffering for storing the journal records. Threads coordinate to allocate and copy into their portion of the buffer. All journal records up to 128 kB are buffered.
WiredTiger syncs the buffered journal records to disk upon many conditions.
Below picture depicts the simple illustration is log write operations






MongoDB Jounal
Below picture depicts the simple illustration is log write operations
MongoDB stats are extracted from serverStatus collection. This Collection has documents that gives 


No comments:

Post a Comment