Writing a Persistent Perl Script

Writing a persistent perl script

If %my_hash in your example have moderate size in its final initialized state, you can simply use one of serialization modules like Storable, JSON::XS or Data::Dumper to keep your data in pre-assembled form between runs. Generate a new file when it is absent and just reload ready content from there when it is present.

Also, you've mentioned that you would call this script in loops. A good strategy would be to not call script right away inside the loop, but build a queue of arguments instead and then pass all of them to script after the loop in single execution. Script would set up its environment and then loop over arguments doing its easy work without need to redo setup steps for each of them.

Write to the Data Section in Perl

You can write to the DATA section with a text editor.

Everything else is dark magic. Store input and program state in a separate file (or a database).

How can I spawn a long running process in a Perl CGI script?

See Watching long processes through CGI.

On the other hand, just managing memory better might also solve your problem. For example, if you are reading entire files into memory at once, try to write the script so that it handles data line-by-line or in fixed sized chunks. Declare your variables in the smallest possible scope.

Try to identify what part of your script is creating the largest memory footprint and post the relevant excerpt in a separate question for more memory management suggestions.

Writing to an address starting with 0x00 with a Perl script

Command line-arguments are NUL-terminated strings. Therefore, you can't pass a string containing a NUL. It would be taken to be the end of the string.

$ perl -e'system("echo", "abc\x00def", "ghi\x00jkl");'
abc ghi

Knowing this, the shell is stripping out the NULs when building the argument.

$ perl -e'printf "%v02X\n", $_ for @ARGV' "$( perl -e'print "\xbf\x84\x04\x08" x 5' )"
BF.84.04.08.BF.84.04.08.BF.84.04.08.BF.84.04.08.BF.84.04.08

$ perl -e'printf "%v02X\n", $_ for @ARGV' "$( perl -e'print "\xbf\x84\x04\x00" x 5' )"
BF.84.04.BF.84.04.BF.84.04.BF.84.04.BF.84.04

auth_overflow2 should be modified to take an escaped form of the address, e.g. the address in hex.

Perl persistent data storage using Data::Dumper

We all have those evenings! Try:

$complex_variable = do $MEMORY || die "Bad data";

looking for light-weight data persistence solution in perl

You have several options:

Storable is a core module and is very efficient. It has some problems with portability, for example someone using an older version of Storable may not be able to read your data. Also, the endianness of the system creating and retrieving that data is important. The network order stoarge options help reduce the portability issues. You can store an arbitrary nested data structure to a file or string and restore it. Storable is supported only by Perl.
YAML is a text based format that works like storable--you can store and restore arbitrary structures to/from YAML files. YAML is nice because there are YAML libraries for several languages. It's not quite as speedy or space efficient as Storable.
JSON is a popular data exchange format with support in many languages. It is very much like YAML in both its strengths and weaknesses.
DBD::SQLite is a database driver for the DBI that allows you to keep a whole relational database in a single file. It is powerful and allows you work with many of the persistence tools that are aimed at other databases like MySQL and Postgres.
DBM::Deep is a convenient and powerful perl only module that allows efficient retrieval and modification of small parts of a large persistent data structures. Almost as easy to use as Storable, but far more efficient when dealing with only small portions of a large data structure.

Update: I realized that I should mention that I have used all of these modules and depending on your particular needs, any of them could be "the right choice".

Perl: Modify data structure in .pm file at runtime

This is a terrible idea. I'm not going to bother answering it, all the solutions will just add an unmaintainable mess on top of whatever unmaintainable mess you're maintaining. There are much better ways to alter a private variable in a .pm file that don't make things worse by trying to parse and alter Perl code.

The simplest thing to do is call getDataStructure (I hope $var and getDataStructure are not the real names) and modify the returned reference.

my $data = getDataStructure;
$data->{new_thing} = { new => "stuff" };

If for some reason this doesn't work for you, since you have the ability to rewrite the file, you should rewrite it to put the data into a separate file, probably in JSON. That JSON file will be much easier to alter.

my $data;
sub getDataStructure {
    $data ||= loadDataStructure("some_config.json");
    return $data;
}

If you can't have a separate file, you can put the JSON in a __DATA__ section at the end of the file and read it from the special DATA filehandle.

my $data;
sub getDataStructure {
    $data ||= loadDataStructure();
    return $data;
}

require JSON;
sub loadDataStructure {
    my $json = join '', <DATA>;
    return JSON->new->decode($json);
}

1;

__DATA__
"xyz": {
    "pqr": ...
}

It is far easier for another program to read everything after __DATA__ as JSON, alter it, and write it back out after __DATA__.

If for some reason THAT doesn't work, then instead of altering $var in place you can simply add to it.

my $var = {
    ...blah blah blah...
};

sub getDataStructure() {
  return $var;
}

# These lines are added at the end.
$var->{new_thing} = { new => "stuff" };
$var->{another_new_thing} = { other => "stuff" };

1;

This is a maintenance nightmare, but it's much easier to append to the end of the .pm file than it is to alter the middle of a data structure. Take care to leave the true value at the end, or eliminate that problem by using true.

Writing a Persistent Perl Script