In theory, if you have a large enough hard drive and free space on the ps3, they might be able write in part of the program about using some of the free space as virtual ram.
Virtual memory isn't really a solution for games.
Virtual memory works best when data gets swapped in and out fairly infrequently - but games generally keep data in RAM on an as-needed basis to begin with, i.e. the contents of a game's memory space is primarily data that it needs to process every frame. Swapping data in and out of virtual memory every frame causes an enormous performance and responsiveness hit.
Just as a grossly simplified example, let's say the PS3 client needs to have 600 MB of data loaded at any given time just to render the graphics and do simulation updates - since the PS3 only has 512 MB of RAM, it has to use 88 MB of virtual memory. This means that every frame, it needs to transfer at least 176 MB of data between hard drive and RAM every frame (writing 88 MB of data being swapped out, than reading another 88MB being swapped in), of which there are 60 each second. This requires the hard drive have a data bandwidth of at least 10 GB/sec.
Hard drives generally transfer about 70 MB/sec... which is about 1/150th of that. Even a USB 3.0 memory stick (out of the question anyway since the PS3 doesn't support USB 3.0) can only transfer 500 MB/sec, which is still too slow by a factor of 20. You can see the problem here.
In short, for games the general rule is, if a given bit of data has been loaded into RAM, in needs to be in physical RAM, not virtual memory.
This is further complicated by the fact that the PS3 client doesn't actually have 512 MB of RAM to work with - really, it has 238 MB of video RAM (256 MB capacity - the PS3 OS takes up 7 MB, and at least 11 MB are needed for the frame buffer) and 213 MB of system RAM (again, the OS itself takes up 43 MB of the 256 MB total).
The net result is that, compared to the PC version, the PS3 version is likely to have some combination of: lower resolution textures, less detailed models, shorter draw distance, and reduced characters shown on-screen.