How to Edit a Big File

Text editor to open big (giant, huge, large) text files

Free read-only viewers:

• Large Text File Viewer (Windows) – Fully customizable theming (colors, fonts, word wrap, tab size). Supports horizontal and vertical split view. Also support file following and regex search. Very fast, simple, and has small executable size.
• klogg (Windows, macOS, Linux) – A maintained fork of glogg. Its main feature is regular expression search. It supports monitoring file changes (like tail), bookmarks, highlighting patterns using different colors, and has serious optimizations built in. But from a UI standpoint, it's rather minimal.
• LogExpert (Windows) – "A GUI replacement for tail." It's really a log file analyzer, not a large file viewer, and in one test it required 10 seconds and 700 MB of RAM to load a 250 MB file. But its killer features are the columnizer (parse logs that are in CSV, JSONL, etc. and display in a spreadsheet format) and the highlighter (show lines with certain words in certain colors). Also supports file following, tabs, multifiles, bookmarks, search, plugins, and external tools.
• Lister (Windows) – Very small and minimalist. It's one executable, barely 500 KB, but it still supports searching (with regexes), printing, a hex editor mode, and settings.

Free editors:

• Your regular editor or IDE. Modern editors can handle surprisingly large files. In particular, Vim (Windows, macOS, Linux), Emacs (Windows, macOS, Linux), Notepad++ (Windows), Sublime Text (Windows, macOS, Linux), and VS Code (Windows, macOS, Linux) support large (~4 GB) files, assuming you have the RAM.
• Large File Editor (Windows) – Opens and edits TB+ files, supports Unicode, uses little memory, has XML-specific features, and includes a binary mode.
• GigaEdit (Windows) – Supports searching, character statistics, and font customization. But it's buggy – with large files, it only allows overwriting characters, not inserting them; it doesn't respect LF as a line terminator, only CRLF; and it's slow.

Builtin programs (no installation required):

• less (macOS, Linux) – The traditional Unix command-line pager tool. Lets you view text files of practically any size. Can be installed on Windows, too.
• Notepad (Windows) – Decent with large files, especially with word wrap turned off.
• MORE (Windows) – This refers to the Windows MORE, not the Unix more. A console program that allows you to view a file, one screen at a time.

Web viewers:

• readfileonline.com – Another HTML5 large file viewer. Supports search.

Paid editors/viewers:

• 010 Editor (Windows, macOS, Linux) – Opens giant (as large as 50 GB) files.
• SlickEdit (Windows, macOS, Linux) – Opens large files.
• UltraEdit (Windows, macOS, Linux) – Opens files of more than 6 GB, but the configuration must be changed for this to be practical: Menu » Advanced » Configuration » File Handling » Temporary Files » Open file without temp file...
• EmEditor (Windows) – Handles very large text files nicely (officially up to 248 GB, but as much as 900 GB according to one report).
• BssEditor (Windows) – Handles large files and very long lines. Don’t require an installation. Free for non commercial use.
• loxx (Windows) – Supports file following, highlighting, line numbers, huge files, regex, multiple files and views, and much more. The free version can not: process regex, filter files, synchronize timestamps, and save changed files.

How do I read a text file of about 2 GB?

Try Glogg.
the fast, smart log explorer.

I have opened log file of size around 2 GB, and the search is also very fast.

Best Free Text Editor Supporting *More Than* 4GB Files?

glogg could also be considered, for a different usage:

Caveat (reported by Simon Tewsi in the comments, Feb. 2013)

One caveat - has two search functions, Main Search and Quick Find.

The lower one, which I assume is Quick Find, is at least an order of magnitude slower than the upper one, which is fast.

looking for fast way to edit a large file in linux

Provided that you know that your header is less than X bytes, you can use dd.
(!) But it only works if both strings have the same length (!)

Lets say, that the header is less that 4096 bytes

dd if=/path/to/file bs=4096 count=1 | sed 's/XXX/YYY/' | dd of=/path/to/file conv=notrunc

You can also do it programmatically, using languages like C,Python,PHP,JAVA etc. The idea is to open the file, read the header, fix it, and write it back.

How to edit a big file

The only way to insert data in the middle of any (huge or small) file (on Linux or POSIX) is to copy that file (into a fresh one, then later rename(2) the copy as the original). So you'll copy its head (up to insertion point), you'll append the data to that copy, and then you copy the tail (after insertion point). You might consider also calling posix_fadvise(2) (or even the Linux specific readahead(2)...) but that does not aleviate the need to copy all the data. mmap(2) might be used e.g. to replace read(2) but whatever you do requires you to copy all the data.

Of course, if it happens that you are replacing a data chunk in the middle of the file by another chunk of the same size (so no real insertion), you can use plain lseek(2) + write(2)

Is the only way to add a line and then move the rest of the file? That sounds expensive.

Yes it is conceptually the only way.

You should consider using something else that a plain textual file: look into SQLite or GDBM (they might be very efficient in your use case). See also this answer. Both provides you with some higher abstraction than POSIX files, so give you the ability to "insert" data (Of course they are still internally based upon and using POSIX files).

Some issues when edit large files more than 100GB

Assuming you are running a relatively recent version of EmEditor

Find (Ctrl+ F): {

Options: Match Case, Close when Finished, (None)

Click [Select All] (should all be selected in your file)

Edit Menu - Advanced - Numbering (or Alt+ N)

First Line:{"id":1

Increment:1

Make sure Decimal is selected

Click [OK]

---

Related Topics