noob: Need Break up large txt file

[gold]

I have a large text file. I want to break the text file in many equal parts based on number of total lines in the txt file. eg. txt file has 1500 lines, then 3 equal small files would have 500 lines in each.

What is the best way to break large txt file into parts base on # of lines?

[marko]

If you have a file "bigfile" then you can use split to cut it up into
500 line chunks:

split -500 bigfile

unfortunately split names the output files in kind of a dumb way:

xaa, xab, xab .......

that is the output files have no naming relation to the input file,
I would have thought bigfile.01, bigfile.02 .... etc would be
more sensible.

"wc" is the handy way to find out how many lines the input
file has to decide how big to make the chunk files:

> wc -l bigfile
14416 bigfile

so if you want three files of about equal size split -d -5000 bigfile
would be a good try with my 14416 line example.
(the -d option sets up the suffix to be two digits which you might like better than the aa, ab, ac, ... default)

see the man page for "split" for more

Mark

[HaydnH]

If you put the filename twice after split it will relate to the filename, you can also split the file by number of bytes, e.g:

[root@localhost test]# dd if=/dev/zero of=./bigfile bs=1024k count=5
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.0595227 s, 88.1 MB/s
[root@localhost test]# split --bytes=1024k bigfile bigfile.
[root@localhost test]# ls -lh
total 11M
-rw-r--r-- 1 root  root  5.0M 2007-12-11 11:00 bigfile
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.aa
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.ab
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.ac
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.ad
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.ae
[root@localhost test]#

And you can rebuild the big text file with the following:

[root@localhost test]# rm -f bigfile
[root@localhost test]# for i in bigfile.*; do cat "$i" >> ./bigfile; done
[root@localhost test]# ls -lh
total 11M
-rw-r--r-- 1 root  root  5.0M 2007-12-11 11:07 bigfile
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.aa
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.ab
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.ac
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.ad
-rw-r--r-- 1 root  root  1.0M 2007-12-11 11:01 bigfile.ae
[root@localhost test]#