High heap memory

Rocky Linux Help & Support
1

We are observing that the dynamic memory getting allocated for C++ STL containers are higher than that in centos. Tried with a basic program having std:vectors and std:list. Memory shown in pmap o/p is higher in rocky. What is contributing to this?

2

What version of CentOS, what version of Rocky, what compiler version in each case, and are all compiler flags and underlying libraries identical in each case?

3

Centos:
Operating System: CentOS Linux 7 (Core)
Kernel: Linux 3.10.0-1160.76.1.el7.x86_64
Compiler: gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC)
Compiler Flags : -O2 -g -fno-omit-frame-pointer -m64 -fstack-protector -D_FORTIFY_SOURCE=2

Rocky:
Operating System: Rocky Linux 8.6 (Green Obsidian)
Kernel: Linux 4.18.0-372.32.1.el8_6.x86_6
Compiler : gcc version 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)
Compiler Flags:-O2 -g -fno-omit-frame-pointer -m64 -fstack-protector -D_FORTIFY_SOURCE=2

Underlying libraries are same.

4

I think I’ve seen similar report before but can’t recall where.

5

Ok, these are two different operating systems with different kernels and different underlying libraries, compiled with different compilers, so yes there’s bound to be huge differences.

1 Like
6

But if the application process is same, should there be increased memory usage?

Tried a simple program using STL, vectors and list, and it is showing difference in memory

in Rocky - 9436 KB
in centos - 8664KB

int main()
{
    vector<string> vc;
    list<string> lc;
    for(int i = 0; i < 100000; ++i)
    {
        string s(to_string(i));
        vc.push_back(s);
        lc.push_back(s);
    }
   return 0;
}

In our project we have multiple vectors, list and maps. Each could be of size 1million. So there will be a huge difference in memory.
Any detail on why the memory requirement is more in rocky?

7

That is not a proper program; vector and list lack template parameters.

#include <vector>
#include <list>
#include <string>
#include <iostream>

int main()
{
  std::vector<std::string> vc;
  std::list<std::string> lc;
  for (int i = 0; i < 100000; ++i)
  {
    std::string s( std::to_string(i) );
    vc.push_back(s);
    lc.push_back(s);
  }
  std::cout << "Size " << vc.size() << " cap " << vc.capacity() << '\n';
}

EL9 x86_64:

$ g++ --version
g++ (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2)

$ /usr/bin/time -v ./heap
Size 100000 cap 131072
...
	Maximum resident set size (kbytes): 12456

EL8 x86_64:

$ g++ --version
g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-15)

$ /usr/bin/time -v ./heap
Size 100000 cap 131072
...
	Maximum resident set size (kbytes): 12324

EL7 x86_64:

$ g++ --version
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)

$ g++  -O2 -g -fno-omit-frame-pointer -m64 -fstack-protector -D_FORTIFY_SOURCE=2 -o heap heap.cpp
heap.cpp: In function ‘int main()’:
heap.cpp:12:20: error: ‘to_string’ is not a member of ‘std’
     std::string s( std::to_string(i) );

Must use C++11 mode too:

$ g++ -std=c++11 -O2 -g -fno-omit-frame-pointer -m64 -fstack-protector -D_FORTIFY_SOURCE=2 -o heap heap.cpp

$ /usr/bin/time -v ./heap
Size 100000 cap 131072
...
	Maximum resident set size (kbytes): 9692

EL7 x86_64, but newer GCC:

$ g++ --version
g++ (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)

$ g++  -O2 -g -fno-omit-frame-pointer -m64 -fstack-protector -D_FORTIFY_SOURCE=2 -o heap heap.cpp
abo2112d-71zk:7:~/test$ /usr/bin/time -v ./heap
Size 100000 cap 131072
...
	Maximum resident set size (kbytes): 14320

If the “Maximum resident set size” is indicative of anything, then binary built with GCC 11 on CentOS 7 uses more memory than same program built by same compiler on Rocky 9.

How has the C++ Standard Library implementation changed between GCC 4.8 and 8.5? That is a question to upstream.
Has the kernel memory management changed between (RHEL versions) “3.10.0” and “4.18.0”? Probably.

Adding vc.reserve(100000); before loop does not change maximum set size but makes program faster as it saves many reallocations.