Mineclonia
/
Mineclonia
Archived
12
17
Fork 13
Code Issues 105 Pull Requests 19 Projects Releases Wiki Activity

Add script to show packets count from client logs #123

Merged
erlehmann merged 1 commits from analyze-packet-spam into master 2021-07-21 09:16:35 +02:00
Conversation 0 Commits 1 Files Changed 1 +60
erlehmann commented 2021-07-17 10:22:51 +02:00
Owner
Copy Link

Problem
TRACKING ISSUE: #111

To address lag issues caused by too many network packets being sent, we need to identify which changes increase the number of sent packets and which decrease it. Doing this now involves using Wireshark and some manual work calculations.

Solution

This patch adds a bourne shell script that calculates per-second packet counts from Minetest log files.

Details

The Minetest engine already logs packet counts every 30 seconds at log level INFO.

The script added in this PR determines the time frame and sums up packets by packet type.
It then calculates the packet count per second by packet type.

Testing Steps
Setup
  1. Check out Mineclonia commit 01c8339f40.
Collect Data
  1. In Bourne shell, run Minetest with timeout 600 minetest --info >log.txt 2>&1 >/dev/null.
  2. Create a new world for map generator v7 with the seed packetspam.
  3. Enter the new world in Mineclonia commit 01c8339f40.
  4. Play Mineclonia for about 10 minutes (or just let it run and do something else).
  5. Verify that a file log.txt has been created and is not empty.
Packet Count Calculation
  1. In Bourne shell, run the tools/analyze-packet-spam script with log.txt given on standard input: ./tools/analyze-packet-spam <log.txt.
  2. Note down the number at the start of the line ending in PACKET_COUNT_TOCLIENT_HUDCHANGE.
Packet Count Verification
  1. In Bourne shell, get the TOCLIENT_HUDCHANGE log lines from log.txt using grep: grep TOCLIENT_HUDCHANGE <log.txt
  2. Add up the numbers at the end of each line to get the total packet count.
  3. Determine the time span for the log by calculating the difference between the timestamp of the last line and the timestamp of the first line and adding 30 seconds.
  4. Verify that the timespan is 9 min 30 s or 570 seconds.
  5. Divide the total packet count by the timespan in seconds.
  6. Verify that the result, when truncated to 1 digit after he decimal dot, matches the result in the Packet Count Calculation section.
To do
  • Fill out issue template
  • Write test plan
  • Verify tests
Problem TRACKING ISSUE: https://git.minetest.land/Mineclonia/Mineclonia/issues/111 To address lag issues caused by too many network packets being sent, we need to identify which changes increase the number of sent packets and which decrease it. Doing this now involves using Wireshark and some manual work calculations. ##### Solution This patch adds a bourne shell script that calculates per-second packet counts from Minetest log files. ##### Details The Minetest engine already logs packet counts every 30 seconds at log level INFO. The script added in this PR determines the time frame and sums up packets by packet type. It then calculates the packet count per second by packet type. ##### Testing Steps ###### Setup 1. Check out Mineclonia commit 01c8339f403154aa5f3e28c45209595aea7c7686. ###### Collect Data 2. In Bourne shell, run Minetest with `timeout 600 minetest --info >log.txt 2>&1 >/dev/null`. 3. Create a new world for map generator v7 with the seed `packetspam`. 4. Enter the new world in Mineclonia commit 01c8339f403154aa5f3e28c45209595aea7c7686. 5. Play Mineclonia for about 10 minutes (or just let it run and do something else). 6. Verify that a file log.txt has been created and is not empty. ###### Packet Count Calculation 1. In Bourne shell, run the `tools/analyze-packet-spam` script with `log.txt` given on standard input: `./tools/analyze-packet-spam <log.txt`. 2. Note down the number at the start of the line ending in PACKET_COUNT_TOCLIENT_HUDCHANGE. ###### Packet Count Verification 1. In Bourne shell, get the TOCLIENT_HUDCHANGE log lines from `log.txt` using `grep`: `grep TOCLIENT_HUDCHANGE <log.txt` 2. Add up the numbers at the end of each line to get the total packet count. 3. Determine the time span for the log by calculating the difference between the timestamp of the last line and the timestamp of the first line and adding 30 seconds. 4. Verify that the timespan is 9 min 30 s or 570 seconds. 5. Divide the total packet count by the timespan in seconds. 6. Verify that the result, when truncated to 1 digit after he decimal dot, matches the result in the `Packet Count Calculation` section. ##### To do - [x] Fill out issue template - [x] Write test plan - [x] Verify tests
erlehmann changed target branch from master to fix-hudchange-spam 2021-07-17 10:23:40 +02:00
erlehmann changed title from Add script to show packets count from client logs to WIP: Add script to show packets count from client logs 2021-07-17 10:23:40 +02:00
erlehmann added a new dependency 2021-07-17 10:24:07 +02:00
#122 HUD/hudbars: Do not send useless HUDCHANGE packets
erlehmann changed target branch from fix-hudchange-spam to master 2021-07-17 10:24:51 +02:00
erlehmann force-pushed analyze-packet-spam from 38b2e70239 to cb5c7346a7 2021-07-18 17:55:12 +02:00 Compare
erlehmann force-pushed analyze-packet-spam from cb5c7346a7 to 01c8339f40 2021-07-19 22:32:10 +02:00 Compare
erlehmann changed title from WIP: Add script to show packets count from client logs to Add script to show packets count from client logs 2021-07-20 11:12:29 +02:00
e reviewed 2021-07-21 05:04:46 +02:00
tools/analyze-packet-spam
@ -0,0 +49,4 @@
*)
PACKET_COUNT_PER_SECOND=$(
printf '1k %s %s /p' "${PACKET_COUNT}" "${DURATION}" \
|dc
e commented 2021-07-21 05:04:46 +02:00
Member

Note: dc isn't specified in POSIX, but bc is.

Note: `dc` isn't specified in POSIX, but `bc` is.
erlehmann commented 2021-07-21 09:20:23 +02:00
Author
Owner

Thanks, I did not know.

Thanks, I did not know.
e approved these changes 2021-07-21 05:19:34 +02:00
e left a comment
Member

Setup

Collect Data

  • In Bourne shell, run Minetest with timeout 600 minetest --info >log.txt 2>&1 >/dev/null.
  • Create a new world for map generator v7 with the seed packetspam.
  • Enter the new world in Mineclonia commit 01c8339f40.
  • Play Mineclonia for about 10 minutes (or just let it run and do something else).
  • Verify that a file log.txt has been created and is not empty.

Packet Count Calculation

  • In Bourne shell, run the tools/analyze-packet-spam script with log.txt given on standard input: ./tools/analyze-packet-spam <log.txt.
  • Note down the number at the start of the line ending in PACKET_COUNT_TOCLIENT_HUDCHANGE.

Packet Count Verification

  • In Bourne shell, get the TOCLIENT_HUDCHANGE log lines from log.txt using grep: grep TOCLIENT_HUDCHANGE <log.txt
  • Add up the numbers at the end of each line to get the total packet count.
  • Determine the time span for the log by calculating the difference between the timestamp of the last line and the timestamp of the first line and adding 30 seconds.
  • Verify that the timespan is 9 min 30 s or 570 seconds.
    (NB: Mine was slightly different (572s), presumably because I took a few seconds longer to navigate the menu or generate the world.)
  • Divide the total packet count by the timespan in seconds.
  • Verify that the result, when truncated to 1 digit after he decimal dot, matches the result in the Packet Count Calculation section.
Setup - [x] Check out Mineclonia commit 01c8339f40. Collect Data - [x] In Bourne shell, run Minetest with `timeout 600 minetest --info >log.txt 2>&1 >/dev/null`. - [x] Create a new world for map generator v7 with the seed `packetspam`. - [x] Enter the new world in Mineclonia commit 01c8339f40. - [x] Play Mineclonia for about 10 minutes (or just let it run and do something else). - [x] Verify that a file `log.txt` has been created and is not empty. Packet Count Calculation - [x] In Bourne shell, run the `tools/analyze-packet-spam` script with `log.txt` given on standard input: `./tools/analyze-packet-spam <log.txt`. - [x] Note down the number at the start of the line ending in `PACKET_COUNT_TOCLIENT_HUDCHANGE`. Packet Count Verification - [x] In Bourne shell, get the `TOCLIENT_HUDCHANGE log` lines from `log.txt` using grep: `grep TOCLIENT_HUDCHANGE <log.txt` - [x] Add up the numbers at the end of each line to get the total packet count. - [x] Determine the time span for the log by calculating the difference between the timestamp of the last line and the timestamp of the first line and adding 30 seconds. - [x] Verify that the timespan is 9 min 30 s or 570 seconds. **(NB: Mine was slightly different (572s), presumably because I took a few seconds longer to navigate the menu or generate the world.)** - [x] Divide the total packet count by the timespan in seconds. - [x] Verify that the result, when truncated to 1 digit after he decimal dot, matches the result in the Packet Count Calculation section.
erlehmann merged commit c1cf50ae15 into master 2021-07-21 09:16:35 +02:00
erlehmann deleted branch analyze-packet-spam 2021-07-21 09:27:01 +02:00
This repo is archived. You cannot comment on pull requests.
Reviewers
No reviewers
e
Labels
Clear labels   
blocker

blocks release until resolved

  
bug

   
code quality

   
confirmed

   
critical

crashes, catastrophic bugs

  
discussion

   
high priority

   
incompatibility

incompatible with mcl2

  
incomplete feature

   
invalid

   
low priority

   
missing feauture

   
needs testing

   
packet spam

   
performance

   
project

meta-issues

  
regression

   
translations

   
unconfirmed
  
in review

   
ready for review
No Label
blocker
bug
code quality
confirmed
critical
discussion
high priority
incompatibility
incomplete feature
invalid
low priority
missing feauture
needs testing
packet spam
performance
project
regression
translations
unconfirmed
in review
ready for review
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Blocks
#122 HUD/hudbars: Do not send useless HUDCHANGE packets
Mineclonia/Mineclonia
Reference: Mineclonia/Mineclonia#123
No description provided.
Delete Branch "analyze-packet-spam"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?