In my research project, I run my java based program on Linux. My program processes text files and generates a csv file as output.
But It crashes after running some time. When I check program log, it shows "Too many open files" error. I have checked that still there is enough free ram available.
When I run the same program on windows it works fine.
Any idea or solution of this problem?
Too many open files" error is specific to Linux. Along-with normal OS files, Linux also treats Devices, Connections, Sockets, user processes and SQL tables as files. In Linux there is a limit of the number of Opened files.
The current personal limit can be checked with "ulimit -n" command.
On the latest linux os versions, the limit will be higher.
Few of the solutions are:
1-System Reboot ( temp Solution )
2-Increasing open files limit in Linux System files.
3-Check the code and fix there, most probably when we open the files in the code, we don't close those properly. or sometimes we mishandle it try/catch part
Further on point 2)-Increasing files limit, You can check system-wide limits of file descriptors in the following file:
cat /proc/sys/fs/file-max
Limit chis can be changed in the below file in nofile param.
/etc/security/limits.conf
A quick solution could be setting limit from console(e.g if you want to set limit to 4096).
ulimit -n 4096
But the limit will reset on reboot. To set the limit permanently, change it in limits.conf mentioned above.
Too many open files" error is specific to Linux. Along-with normal OS files, Linux also treats Devices, Connections, Sockets, user processes and SQL tables as files. In Linux there is a limit of the number of Opened files.
The current personal limit can be checked with "ulimit -n" command.
On the latest linux os versions, the limit will be higher.
Few of the solutions are:
1-System Reboot ( temp Solution )
2-Increasing open files limit in Linux System files.
3-Check the code and fix there, most probably when we open the files in the code, we don't close those properly. or sometimes we mishandle it try/catch part
Further on point 2)-Increasing files limit, You can check system-wide limits of file descriptors in the following file:
cat /proc/sys/fs/file-max
Limit chis can be changed in the below file in nofile param.
/etc/security/limits.conf
A quick solution could be setting limit from console(e.g if you want to set limit to 4096).
ulimit -n 4096
But the limit will reset on reboot. To set the limit permanently, change it in limits.conf mentioned above.
Make sure you close each file after you have finished processing it! And also, when something goes wrong (exception) during the processing stage. I doubt you need to have more than 1024 files opened simultaneously. Do you?
Too many open files is the maximum number of file descriptors the process has out of system limits.
use this command to check maximum:
ulimit -a | grep open
use this command to set maximum:
ulimit -n 524288
ulimit -a lists all of your development limits in Linux. They can all be increased:
[charles.johnson@charles-johnson main]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 38903
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 4096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[charles.johnson@charles-johnson main]$
You can change any limit up to a "hard" limit ceiling with the command -S for the switch listed in the parenthesis, example for open files:
ulimit -Sn unlimited
There is a configuration file you can set for this more permanently, otherwise it reverts on your next login.
- Badges
- Science topic
- Similar topics
- Biological Science
- Kinesiology
- Running