Fli
03-18-2021, 03:08 PM
I have seen this more than one that when copying files to a FAT32 formatted USB flash drive via Dolphin file manager interface, the process was shown as 100% complete, but staled or staled in the middle of the transfer.
UPDATE: the cause may be bad flash drive or bad USB port (someone advised trying to erase the drive: dd if=/dev/zero of=/dev/sdy bs=4k status=progress)
Other drive with FAT32 in different port is working well.
Some more similar issues:
https://unix.stackexchange.com/a/181236/46902
https://forum.manjaro.org/t/the-pernicious-usb-stick-stall-problem/52297
https://forum.manjaro.org/t/file-copy-on-a-usb-2-slows-down-to-a-halt/36316 (https://forum.manjaro.org/t/the-pernicious-usb-stick-stall-problem/52297)
https://forum.manjaro.org/t/copying-progress-bar-hangs-at-100-when-copying-to-thumb-drive/37108
-----
Also i see some CPU iowait reported by htop, i think likely due to this.
And some processes reported to do IO by iotop command:
3119951 be/4 username 0.00 B 0.00 B 0.00 % 99.99 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinJKiEwu.328.slave-socket
3104486 be/4 username 0.00 B 0.00 B 0.00 % 99.99 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinNLnSjj.318.slave-socket
3125460 be/4 username 0.00 B 0.00 B 0.00 % 92.69 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinIZYzMd.332.slave-socket
3125462 be/4 username 0.00 B 0.00 B 0.00 % 89.03 % thumbnail.so [kdeinit5] thumbnail local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinNqLUwp.333.slave-socket
3124583 be/4 username 0.00 B 0.00 B 0.00 % 88.98 % thumbnail.so [kdeinit5] thumbnail local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinydSXye.331.slave-socket
3012539 be/4 root 0.00 B 0.00 B 0.00 % 88.87 % [kworker/u64:9+flush-8:0]
3119702 be/4 username 0.00 B 0.00 B 0.00 % 88.94 % thumbnail.so [kdeinit5] thumbnail local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinWVywVM.327.slave-socket
3116983 be/4 username 0.00 B 0.00 B 0.00 % 88.93 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinlObCCE.325.slave-socket
3127011 be/4 username 0.00 B 0.00 B 0.00 % 89.03 % thumbnail.so [kdeinit5] thumbnail local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinSjXRpH.334.slave-socket
3120474 be/4 username 0.00 B 0.00 B 0.00 % 78.23 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinQbSKLN.330.slave-socket
3017771 be/4 root 0.00 B 0.00 B 0.00 % 78.14 % [kworker/u64:14+events_freezable_power_]
Glances shows the drive doing like 100K transfer per second.
The Dolphin is hardly responding but some of the transfer windows in it shows some very slow transfer activity at times.
I have found some suggestion regarding kernel parameters:
vm.dirty_background_bytes
vm.dirty_bytes
https://archived.forum.manjaro.org/t/decrease-dirty-bytes-for-more-reliable-usb-transfer/62513 (here more details and how to make it persistent: https://gist.github.com/2E0PGS/f63544f8abe69acc5caaa54f56efe52f )
Someone was playing with these parameters here https://forum.manjaro.org/t/my-system-tweaks-to-achieve-better-performances-based-on-my-needs/43808 , but for me it seems difficult to mess with that
$ sudo sysctl -a | grep dirty
sysctl: reading key "kernel.spl.hostid"
vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 6000
vm.dirty_ratio = 20
vm.dirty_writeback_centisecs = 6000
vm.dirtytime_expire_seconds = 43200
so i ask your opinion here please.
My system has rather high amount of free memory.
If i have 64GB ram, i tried to change like this:
su
sysctl -a | grep dirty;echo $((64*1024*1024)) > /proc/sys/vm/dirty_background_bytes;echo $((192*1024*1024)) > /proc/sys/vm/dirty_bytes;sysctl -a | grep dirty
exit
It changed the values, yet when i copy like three files at once to the flash disk, the average speed per one file is 150KB/s and after couple of minutes becoming idle.
i am unable to reset the value back to zero (0 is invalid argument), so i will wait till the next restart and do not know how to solve this. Some people suggested rsync may work, but i want GUI transfers.
UPDATE: the cause may be bad flash drive or bad USB port (someone advised trying to erase the drive: dd if=/dev/zero of=/dev/sdy bs=4k status=progress)
Other drive with FAT32 in different port is working well.
Some more similar issues:
https://unix.stackexchange.com/a/181236/46902
https://forum.manjaro.org/t/the-pernicious-usb-stick-stall-problem/52297
https://forum.manjaro.org/t/file-copy-on-a-usb-2-slows-down-to-a-halt/36316 (https://forum.manjaro.org/t/the-pernicious-usb-stick-stall-problem/52297)
https://forum.manjaro.org/t/copying-progress-bar-hangs-at-100-when-copying-to-thumb-drive/37108
-----
Also i see some CPU iowait reported by htop, i think likely due to this.
And some processes reported to do IO by iotop command:
3119951 be/4 username 0.00 B 0.00 B 0.00 % 99.99 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinJKiEwu.328.slave-socket
3104486 be/4 username 0.00 B 0.00 B 0.00 % 99.99 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinNLnSjj.318.slave-socket
3125460 be/4 username 0.00 B 0.00 B 0.00 % 92.69 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinIZYzMd.332.slave-socket
3125462 be/4 username 0.00 B 0.00 B 0.00 % 89.03 % thumbnail.so [kdeinit5] thumbnail local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinNqLUwp.333.slave-socket
3124583 be/4 username 0.00 B 0.00 B 0.00 % 88.98 % thumbnail.so [kdeinit5] thumbnail local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinydSXye.331.slave-socket
3012539 be/4 root 0.00 B 0.00 B 0.00 % 88.87 % [kworker/u64:9+flush-8:0]
3119702 be/4 username 0.00 B 0.00 B 0.00 % 88.94 % thumbnail.so [kdeinit5] thumbnail local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinWVywVM.327.slave-socket
3116983 be/4 username 0.00 B 0.00 B 0.00 % 88.93 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinlObCCE.325.slave-socket
3127011 be/4 username 0.00 B 0.00 B 0.00 % 89.03 % thumbnail.so [kdeinit5] thumbnail local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinSjXRpH.334.slave-socket
3120474 be/4 username 0.00 B 0.00 B 0.00 % 78.23 % file.so [kdeinit5] file local:/run/user/1000/klauncherMrCXav.1.slave-socket local:/run/user/1000/dolphinQbSKLN.330.slave-socket
3017771 be/4 root 0.00 B 0.00 B 0.00 % 78.14 % [kworker/u64:14+events_freezable_power_]
Glances shows the drive doing like 100K transfer per second.
The Dolphin is hardly responding but some of the transfer windows in it shows some very slow transfer activity at times.
I have found some suggestion regarding kernel parameters:
vm.dirty_background_bytes
vm.dirty_bytes
https://archived.forum.manjaro.org/t/decrease-dirty-bytes-for-more-reliable-usb-transfer/62513 (here more details and how to make it persistent: https://gist.github.com/2E0PGS/f63544f8abe69acc5caaa54f56efe52f )
Someone was playing with these parameters here https://forum.manjaro.org/t/my-system-tweaks-to-achieve-better-performances-based-on-my-needs/43808 , but for me it seems difficult to mess with that
$ sudo sysctl -a | grep dirty
sysctl: reading key "kernel.spl.hostid"
vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 6000
vm.dirty_ratio = 20
vm.dirty_writeback_centisecs = 6000
vm.dirtytime_expire_seconds = 43200
so i ask your opinion here please.
My system has rather high amount of free memory.
If i have 64GB ram, i tried to change like this:
su
sysctl -a | grep dirty;echo $((64*1024*1024)) > /proc/sys/vm/dirty_background_bytes;echo $((192*1024*1024)) > /proc/sys/vm/dirty_bytes;sysctl -a | grep dirty
exit
It changed the values, yet when i copy like three files at once to the flash disk, the average speed per one file is 150KB/s and after couple of minutes becoming idle.
i am unable to reset the value back to zero (0 is invalid argument), so i will wait till the next restart and do not know how to solve this. Some people suggested rsync may work, but i want GUI transfers.