How to Write a Perl, Python, or Ruby Program to Change the Memory of Another Process on Windows

How to write a Perl, Python, or Ruby program to change the memory of another process on Windows?

I initially thought this was not possible but after seeing Brian's comment, I searched CPAN and lo and behold, there is Win32::Process::Memory:

C:\> ppm install Win32::Process::Info
C:\> ppm install Win32::Process::Memory

The module apparently uses the ReadProcessMemory function: Here is one of my attempts:

#!/usr/bin/perl
use strict; use warnings;

use Win32;
use Win32::Process;
use Win32::Process::Memory;

my $process;

Win32::Process::Create(
    $process,
    'C:/opt/vim/vim72/gvim.exe',
    q{},
    0,
    NORMAL_PRIORITY_CLASS,
    q{.}
) or die ErrorReport();

my $mem = Win32::Process::Memory->new({
    pid => $process->GetProcessID(),
    access => 'read/query',
});

$mem->search_sub( 'VIM', sub {
    print $mem->hexdump($_[0], 0x20), "\n";
});

sub ErrorReport{
    Win32::FormatMessage( Win32::GetLastError() );
}

END { $process->Kill(0) if $process }

Output:

C:\Temp> proc
0052A580 : 56 49 4D 20 2D 20 56 69 20 49 4D 70 72 6F 76 65 : VIM - Vi IMprove
0052A590 : 64 20 37 2E 32 20 28 32 30 30 38 20 41 75 67 20 : d 7.2 (2008 Aug

0052A5F0 :       56 49 4D 52 55 4E 54 49 4D 45 3A 20 22 00 :   VIMRUNTIME: ".
0052A600 : 20 20 66 61 6C 6C 2D 62 61 63 6B 20 66 6F 72 20 :   fall-back for
0052A610 : 24 56                                           : $V

Grabbing memory from another process

You should be able to use the ReadProcessMemory function.

See also How to write a Perl, Python, or Ruby program to change the memory of another process on Windows?

Please introduce a multi-processing library in Perl or Ruby

With Perl, you have options. One option is to use processes as below. I need to look up how to write the analogous program using threads but http://perldoc.perl.org/perlthrtut.html should give you an idea.

#!/usr/bin/perl

use strict;
use warnings;

use Parallel::ForkManager;

my @data = (0 .. 19);

my $pm = Parallel::ForkManager->new(4);

for my $n ( @data ) {
    my $pid = $pm->start and next;
    warn sprintf "%d^3 = %d\n", $n, slow_cube($n);
    $pm->finish;
}

sub slow_cube {
    my ($n) = @_;

    sleep 1;
    return $n * $n * $n;
}

__END__

The following version using threads does not use a limit on the number of threads created (because I do not know how):

#!/usr/bin/perl

use strict;
use warnings;

use threads;

my @data = (0 .. 19);
my @threads = map { 
    threads->new( {context => 'list' }, \&slow_cube, $_ )
} @data;

for my $thr ( @threads ) {
    my ( $n, $ncubed ) = $thr->join;
    print "$n^3 = $ncubed\n";
}

sub slow_cube {
    my ($n) = @_;

    sleep 1;
    return $n, $n * $n * $n;
}

__END__

Interestingly:

TimeThis :  Command Line :  t.pl
TimeThis :  Elapsed Time :  00:00:01.281

Perl out of memory with large text file

Handle the file line by line:

while ( my $file = $doc_it->() ) { # go through all documents found
    print "Stripping $file\n";

    open (my $infh, "<", $file) or die "Can't open $file for read: $!";
    open (my $outfh, ">", $file . ".tmp") or die "Can't open $file.tmp for write: $!";

    while (<$infh>) {
       if ( /<DOCUMENT>/ ) {
           # append the next line to test for TYPE
           $_ .= <$infh>;
           if (/<TYPE>EX-/) {
              # document type is excluded, now loop through 
              # $infh until the closing tag is found.
              while (<$infh>) { last if m|</DOCUMENT>|; }

              # jump back to the <$infh> loop to resume
              # processing on the next line after </DOCUMENT>
              next;
           }
           # if we've made it this far, the document was not excluded
           # fall through to print both lines
       }
       print $outfh $_;
    }

    close $outfh or die "Cannot close $file: $!";
    close $infh or die "Cannot close $file: $!";
    unlink $file;
    rename $file.'.tmp', $file; 
}

Why are scripting languages (e.g. Perl, Python, and Ruby) not suitable as shell languages?

There are a couple of differences that I can think of; just thoughtstreaming here, in no particular order:

1. Python & Co. are designed to be good at scripting. Bash & Co. are designed to be only good at scripting, with absolutely no compromise. IOW: Python is designed to be good both at scripting and non-scripting, Bash cares only about scripting.

2. Bash & Co. are untyped, Python & Co. are strongly typed, which means that the number 123, the string 123 and the file 123 are quite different. They are, however, not statically typed, which means they need to have different literals for those, in order to keep them apart.
   
   Example:

                   | Ruby             | Bash    
   -----------------------------------------
   number          | 123              | 123
   string          | '123'            | 123
   regexp          | /123/            | 123
   file            | File.open('123') | 123
   file descriptor | IO.open('123')   | 123
   URI             | URI.parse('123') | 123
   command         | `123`            | 123
   

3. Python & Co. are designed to scale up to 10000, 100000, maybe even 1000000 line programs, Bash & Co. are designed to scale down to 10 character programs.

4. In Bash & Co., files, directories, file descriptors, processes are all first-class objects, in Python, only Python objects are first-class, if you want to manipulate files, directories etc., you have to wrap them in a Python object first.

5. Shell programming is basically dataflow programming. Nobody realizes that, not even the people who write shells, but it turns out that shells are quite good at that, and general-purpose languages not so much. In the general-purpose programming world, dataflow seems to be mostly viewed as a concurrency model, not so much as a programming paradigm.

I have the feeling that trying to address these points by bolting features or DSLs onto a general-purpose programming language doesn't work. At least, I have yet to see a convincing implementation of it. There is RuSH (Ruby shell), which tries to implement a shell in Ruby, there is rush, which is an internal DSL for shell programming in Ruby, there is Hotwire, which is a Python shell, but IMO none of those come even close to competing with Bash, Zsh, fish and friends.

Actually, IMHO, the best current shell is Microsoft PowerShell, which is very surprising considering that for several decades now, Microsoft has continually had the worst shells evar. I mean, COMMAND.COM? Really? (Unfortunately, they still have a crappy terminal. It's still the "command prompt" that has been around since, what? Windows 3.0?)

PowerShell was basically created by ignoring everything Microsoft has ever done (COMMAND.COM, CMD.EXE, VBScript, JScript) and instead starting from the Unix shell, then removing all backwards-compatibility cruft (like backticks for command substitution) and massaging it a bit to make it more Windows-friendly (like using the now unused backtick as an escape character instead of the backslash which is the path component separator character in Windows). After that, is when the magic happens.

They address problem 1 and 3 from above, by basically making the opposite choice compared to Python. Python cares about large programs first, scripting second. Bash cares only about scripting. PowerShell cares about scripting first, large programs second. A defining moment for me was watching a video of an interview with Jeffrey Snover (PowerShell's lead designer), when the interviewer asked him how big of a program one could write with PowerShell and Snover answered without missing a beat: "80 characters." At that moment I realized that this is finally a guy at Microsoft who "gets" shell programming (probably related to the fact that PowerShell was neither developed by Microsoft's programming language group (i.e. lambda-calculus math nerds) nor the OS group (kernel nerds) but rather the server group (i.e. sysadmins who actually use shells)), and that I should probably take a serious look at PowerShell.

Number 2 is solved by having arguments be statically typed. So, you can write just 123 and PowerShell knows whether it is a string or a number or a file, because the cmdlet (which is what shell commands are called in PowerShell) declares the types of its arguments to the shell. This has pretty deep ramifications: unlike Unix, where each command is responsible for parsing its own arguments (the shell basically passes the arguments as an array of strings), argument parsing in PowerShell is done by the shell. The cmdlets specify all their options and flags and arguments, as well as their types and names and documentation(!) to the shell, which then can perform argument parsing, tab completion, IntelliSense, inline documentation popups etc. in one centralized place. (This is not revolutionary, and the PowerShell designers acknowledge shells like the DIGITAL Command Language (DCL) and the IBM OS/400 Command Language (CL) as prior art. For anyone who has ever used an AS/400, this should sound familiar. In OS/400, you can write a shell command and if you don't know the syntax of certain arguments, you can simply leave them out and hit F4, which will bring a menu (similar to an HTML form) with labelled fields, dropdown, help texts etc. This is only possible because the OS knows about all the possible arguments and their types.) In the Unix shell, this information is often duplicated three times: in the argument parsing code in the command itself, in the bash-completion script for tab-completion and in the manpage.

Number 4 is solved by the fact that PowerShell operates on strongly typed objects, which includes stuff like files, processes, folders and so on.

Number 5 is particularly interesting, because PowerShell is the only shell I know of, where the people who wrote it were actually aware of the fact that shells are essentially dataflow engines and deliberately implemented it as a dataflow engine.

Another nice thing about PowerShell are the naming conventions: all cmdlets are named Action-Object and moreover, there are also standardized names for specific actions and specific objects. (Again, this should sound familar to OS/400 users.) For example, everything which is related to receiving some information is called Get-Foo. And everything operating on (sub-)objects is called Bar-ChildItem. So, the equivalent to ls is Get-ChildItem (although PowerShell also provides builtin aliases ls and dir – in fact, whenever it makes sense, they provide both Unix and CMD.EXE aliases as well as abbreviations (gci in this case)).

But the killer feature IMO is the strongly typed object pipelines. While PowerShell is derived from the Unix shell, there is one very important distinction: in Unix, all communication (both via pipes and redirections as well as via command arguments) is done with untyped, unstructured strings. In PowerShell, it's all strongly typed, structured objects. This is so incredibly powerful that I seriously wonder why noone else has thought of it. (Well, they have, but they never became popular.) In my shell scripts, I estimate that up to one third of the commands is only there to act as an adapter between two other commands that don't agree on a common textual format. Many of those adapters go away in PowerShell, because the cmdlets exchange structured objects instead of unstructured text. And if you look inside the commands, then they pretty much consist of three stages: parse the textual input into an internal object representation, manipulate the objects, convert them back into text. Again, the first and third stage basically go away, because the data already comes in as objects.

However, the designers have taken great care to preserve the dynamicity and flexibility of shell scripting through what they call an Adaptive Type System.

Anyway, I don't want to turn this into a PowerShell commercial. There are plenty of things that are not so great about PowerShell, although most of those have to do either with Windows or with the specific implementation, and not so much with the concepts. (E.g. the fact that it is implemented in .NET means that the very first time you start up the shell can take up to several seconds if the .NET framework is not already in the filesystem cache due to some other application that needs it. Considering that you often use the shell for well under a second, that is completely unacceptable.)

The most important point I want to make is that if you want to look at existing work in scripting languages and shells, you shouldn't stop at Unix and the Ruby/Python/Perl/PHP family. For example, Tcl was already mentioned. Rexx would be another scripting language. Emacs Lisp would be yet another. And in the shell realm there are some of the already mentioned mainframe/midrange shells such as the OS/400 command line and DCL. Also, Plan9's rc.

How to get started multithreading in Perl

Since your threads are simply going to launch a process and wait for it to end, best to bypass the middlemen and just use processes. Unless you're on a Windows system, I'd recommend Parallel::ForkManager for your scenario.

use Parallel::ForkManager qw( );

use constant MAX_PROCESSES => ...;

my $pm = Parallel::ForkManager->new(MAX_PROCESSES);

my @qfns = ...;

for my $qfn (@qfns) {
   my $pid = $pm->start and next;
   exec("extprog", $qfn)
      or die $!;
}

$pm->wait_all_children();

If you wanted you avoid using needless intermediary threads in Windows, you'd have to use something akin to the following:

use constant MAX_PROCESSES => ...;

my @qfns = ...;

my %children;
for my $qfn (@qfns) {
   while (keys(%children) >= MAX_PROCESSES) {
      my $pid = wait();
      delete $children{$pid};
   }

   my $pid = system(1, "extprog", $qfn);
   ++$children{$pid};
}

while (keys(%children)) {
   my $pid = wait();
   delete $children{$pid};
}

---

Related Topics