MapC tool for large files !!!! solution - Printable Version

+- All-Aircraft-Simulations (https://allaircraftsimulations.com)
+-- Forum: IL2 MAP MODS Download & Discussion (https://allaircraftsimulations.com/forumdisplay.php?fid=269)
+--- Forum: OLD IL2 MAP MODS Discussion & WIP-MAP-MODS (https://allaircraftsimulations.com/forumdisplay.php?fid=307)
+--- Thread: MapC tool for large files !!!! solution (/showthread.php?tid=53239)

---

- fabianfred - 02.04.2008

....just that mapC is made in Microdem at 75pixels...and map T/H at 300

map covers the same area...just 3:4 scale

---

- Boo - 02.04.2008

So it sounds like I am trying to make the biggest map yet then. Ouch! (EDIT : No the East Java map is bigger)

I could trim it up some, the 6 downloaded GeoTiff files I put together extended up into France and down into Africa a bit as well as past Mallorca in the East. It is a lot easier using whole downloaded GeoTiff files, although I could knock them to down to 75-90% size, but would rather not. I am not a pro at PaintShop or Gimp, so it is difficult figuring out how to layer the 4 necessary maps and crop them all at once to the same size.

If anyone wants to make a tutorial for that I would be greatly appreciative, (hint hint )

What really struck me making this map, not being a Spaniard, is that I never really realized how big Spain is even though I have been studying it for a while. I always made my measurements from centrally located Madrid so I never measured anything farther than 5-600 kilometers and my brain never put it together that 500K each way is 1000K coast to coast. :roll: duh!

---

- Boo - 02.04.2008

I did a little math on some of those sizes and it looks to be so far that 500square kilometers is the biggest map yet or around 200 million pixels.

As dimensions go up the square area grows at a 4 fold rate, quickly making maps beyond 500Sq Km in area apparently too big for the IL2 engine.

It would intereting to see if someone could break this limit, perhaps using a 64 bit system? I am curious whether it fails due to the size of the map file plus windows and all the other programs consuming all available RAM up to the 32 bit XP limit.

---

- Boo - 03.04.2008

It's my lucky night. Big time success!
1100Km x 900Km for 990,000sq Km
Load time was 15 seconds (although it might have been longer the first time I loaded) and fps was normal running at 1920x1080 (of course there are no objects yet)
(Core2 Duo running at 3.4Ghz, 4 gigs RAM and a standard 7200 RPM hdd, 8800GTX video)

Still having some problems with the FMB+ and actors static though. If I try to load with the modified FMB it crashes with a memory error, if I use normal FMB it loads fine. I copied a basic actors.static from my other map into the folder to get it to save and test out until I figure out this FMB problem.


MY AllSPain map in FMB, can't fit it all in the FMB!



The Rock! but too bad Africa is not there on the other side. I had to cut it out, although there is a good sized chunk in the SE corner South of Mallorca that is about 250x70Km in size (bigger than our old IL2 Desert map even)



Tooling around in an A-20, bombers are nice for sightseeing, turn on 8x speed and level stab and off she goes. Notice I turned on framerates, and that is at 8x speed still.

---

- GerritJ9 - 03.04.2008

What OS are you using, Boo- 32 bit or 64 bit? I'm running W2000 Pro on my "experimental" computer, which is where I test the IL2 mods. Additionally, this system isn't exactly the most modern: AMD Athlon XP2500+, 1.5 GB DDR-400, 80 GB IDE HDD, Albatron KX600 Pro MB.
If I can get the East Java map up and running at 990000 km2, I only have to remove a small slice. The absolute minimum I can crop the map to without losing any significant content is about 850,000 km2 I think- though I'll have to investigate as to what's possible. Otherwise, it's try again on my more modern rig- AMD 64X2 3800, 2 GB DDR-800, 160 GB SATA HDD, MSI K9AGM2-L MB, XP Pro. If the IL2 engine's limit is linear dimension rather than area, I'm sunk. .............. unless someone who will deserve to go far discovers how to remove that limit.

---

- GerritJ9 - 03.04.2008

Just checked on the Italy-Greece-North Africa map- at a rough guess, this is 1200 x 1200 km approximately, or 1,440,000 km2!!!!!!!!! So area in itself is clearly not the limit- but a linear dimension might be.........
Either that, or my ancient test rig is simply not up to handling large maps.

---

- Hellzone - 03.04.2008

hmm intresting, my pc should beable to eat this up

---

- MiWa - 03.04.2008

Memory is mostly consumed by the 'chopped' map_c.tga. I think this file is by far the largest.
If you have all sea and a couple of islands you can have a huge map because it is mostly sea and map_c.tga
is compressed significantly. It is the shores and rivers that make compressed map_c.tga big.
In other words, blocks which contain both water and land can't be compressed.
So I doubt that actual dimensions of the map are the key, it is the byte size of the compressed
map_c.tga which causes problems.

---

- GerritJ9 - 03.04.2008

The preprocessed map_c for East Java is 409 MB, when run through the map_c tool it is reduced to less than 20 MB (have to check for the exact figure). I'd estimate that the Italy-Greece-North Africa map contains far more land mass than my East Java map does, in which case the limit could well be my test/development computer rather than the map.

---

- Hellzone - 03.04.2008

400mb hock: dear god

---

- MiWa - 03.04.2008

GerritJ9 Wrote:The preprocessed map_c for East Java is 409 MB, when run through the map_c tool it is reduced to less than 20 MB (have to check for the exact figure). I'd estimate that the Italy-Greece-North Africa map contains far more land mass than my East Java map does, in which case the limit could well be my test/development computer rather than the map.

Pure land mass (no rivers or shores) is also compressed easily. This might be the case
for Italy-Greece-North Africa. However, now that you mention it, I'm not sure if the map_c
is decompressed before it is loaded into memory or it is decoded on the fly based
on player's position. The latter would probably cause some performance hits, but
it would allow for far larger in game maps.
Judging by your file sizes, I'm afraid they picked the first option.

---

- GerritJ9 - 03.04.2008

After running the map_c tool, the map_c is reduced to 16.2 MB. Given the estimated size of the Italy-Greece-North Africa map, its processed map_c must be larger than my 16.2 MB. So the size of map_c in itself is probably not an issue. Furthermore, the Spain map posted above is not that much smaller in area- and has MUCH more land mass.
Strangely, I get different text lines at the bottom of the FMB screen: First time I open FMB after starting IL2, then (try to) load the East Java map, I get: "World.land().LoadMap()error: java.lang.Runtime exception: Landscape 'EastJava_NEI/load.ini' loading error. If I then exit and then reopen FMB, I get the small rectangle "Loading EastJava-NEI".
The same thing happens when I open FMB editor: when I start IL2 and open FMB editor for the first time, then try to load the East Java map, I get the "World.land()........." text. Closing and then restarting FMB editor then results in "Loading EastJava-NEI". Perhaps it has to do with the absence of map_F and ed_m01/02/03???????

---

- Boo - 03.04.2008

I am using 32bit XP Pro.

The map loads intially (with memory cache cleared) in 28 seconds for me. Map reloads are about 12 seconds.

I also used the CMAP transform tool this time before dissecting, not sure if that made a difference. Although this map seems to have less anomolies than my previous maps. Actually I did not spot any problems in my high speed cruise over around 600 km of coastline besides textures and rivers and such needing tweaking.

Has anyone tried using RLE texture compression on any of these files, if not I will make a copy of some of them and see if they work when substituted.

File sizes are:
map_c 12.7 megs
map_h 13.5 megs
map_F 94 megs
ed_m01 70.5 megs

The good news is that compressed into a RAR file for uploading it is only 23 megabytes.

---

- FA_Cheech - 03.04.2008

After my personal experience working on The Slot, I'm not sure pixel size will be the issue.

How many 14 hr. days you want to spend with your head in the FMB, and your wife or girlfriends patience with the obsession will probably be the limiting factors.

---

- Boo - 03.04.2008

Well I remade the map at 75% and the FMB+ worked fine. Perhaps I will try 90% tomorrow. I doubt my files are not made incorrectly for it to work with normal FMB and not with the modded FMB tools. Perhaps I am at a memory limit and loading the enhanced FMB puts me over the limit?

Yeah Cheech, for this scale of map you could never poulate the whole thing. I need to think about how I want to do it.

I like using the whole map as a base instead of cutting the country into chunks. It looks good and adds a touch of realism when mapping out your missions to see the familiar shape of the country and know exactly where you are in relation to it. I have always felt diconnected from the world playing the small IL2 maps that show a non-descript rectangle of land in the map every mission.

There seems to not be much of a memory hit at all (at least in my fast system) in loading or playing with a huge map. So either I could minimally cover many sections of map the map that saw aerial combat during the SCW, or I could make multiple maps using the same full scale map and concentrate on populating certain sectors of it for different phases of the war.

Once I get this FMB full scale issue realized (or learn I have to go with a smaller scale) I would like to post it and have people with lesser systems try it out for performance.

I will also be relying on Delco's auto-pop to help populate this map, taking much of the work out of it.

Now if only I could find somone able to rework many of our existing planes into new planes, early versions detuned to SCW era levels of performance......

..and my apologies to any Portugese flyers out there as I do not plan on populating that country at all since I must conserve resources and this is for the SCW.

---