Multi-threading limit in AWS virtual windows

We ask the operating system how many processors there are, so this must be some limit imposed by the virtual machine. Probably intended to keep you from bringing the system to its knees. This is sometimes an adjustable parameter in the virtual machine.

Windows Server 2019 datacenter

One of the vCPUs i've tried:

c5n.18xlarge: Xeon platinuum 8124M 18 cores, 36 logical processors

It *seems* to only be able to access 17 threads when on a similar physical machine it can use 35 threads.  (total logical processors - 1)

Maybe I'm looking at the wrong place, but looking at https://aws.amazon.com/ec2/pricing/on-demand/, if I select "Windows" for the OS,  c5n.18xlarge shows up as having 72 virtual CPUs. So I'm not sure why you say you have 18 cores and 36 logical processors. Is there any chance your instance has the number of vCPUs limited somehow as described in https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-optimize-cpu.html?

Igor currently limits multithreading to 99 threads, regardless of how many actual threads your machine can support. I don't think that's the issue here since it does not seem that your hardware would support > 99 threads.

What do you get if you execute:

Unless you have previously executed SetIgorOption SetNumProcessors=x where x is some value, then ThreadProcessorCount just calls GetNativeSystemInfo and returns the value of dwNumberOfProcessors. That member is documented as "The number of logical processors in the current group." According to https://docs.microsoft.com/en-us/windows/win32/procthread/processor-groups, a processor group can have at most 64 logical processors.

I don't believe that Igor supports using more than 1 thread group (*), so if your "machine" really has 72 virtual CPUs, I wouldn't expect Igor to be able to support more than 64 threads, but I don't know why you would only be able to access 17 threads. But you haven't explained how you came up with that number either.

(*) It's possible that a command that uses tbb (marked as "Automatically Multithreaded" in the help browser) might be able to take advantage of multiple processor groups.

Thank you for the response.  I was not being very clear; i will try to add more details:

All virtual servers we are testing have DUAL SOCKET virtual processors - two cpu 'nodes'.  for some reason, Igor can only utilize one virtual processor, not both.  On similar physical computers , this is not an issue - Igor can use all the cpu cores and 'threads' even if it's on a dual processor system.

I do not have low level access to the AWS virtual environment; i only have the info that the OS is telling me.

I starting this test again on a different Windows 2019 server with TWO of these processors totalling 96 available threads (AWS c5.24xlarge):

Intel Processor
    Cores    24
    Threads    48
    Name    Intel Processor
    Specification    Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz
    Family    6
    Extended Family    6
    Model    5
    Extended Model    55
    Stepping    7
    Instructions    MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, Intel 64
    Virtualization    Unsupported
    Hyperthreading    Supported, Enabled

The code is using ThreadProcessorCount  and ThreadGroupCreate()

I will keep testing more types of virtual CPUs but it would be nice to get more then 48 threads going at the same time when the OS seems to think there are 96 available threads (even if it's maxed out at 64 - that's ok).
   

What you report in your last comment is described at https://chrisgreendevelopmentblog.wordpress.com/2017/08/29/thread-pools-and-windows-processor-groups/. Search for "Long story short:"

Any process has access to all the threads in a single processor group. Since your machine has > 64 threads, it has 2 processor groups, each with 48 threads. So any single instance of Igor has access to 48 threads. If you were running two Igor process at the same time, and if the OS assigned each process to a different processor group (I don't know how you could test for this fact), then you could theoretically use 96 threads at a time.

As far as I know there are no machines available with >64 threads that are not outrageously expensive. OK, the Ryzen Threadripper 3990X is 64-core/128-thread and about $4000 (just for the CPU), which isn't quite outrageously expensive, but it's pretty close. With covid-19, it's going to be pretty difficult to get parts to build a machine with that processor.

That's a long way of saying that your use case is currently very extreme and probably not something we can add support for, at least in the near term.

If you want to provide more information about exactly what you're doing, and where you think the bottlenecks might be, we may be able to help you write more performant code even if it can't take advantage of *all* of your cores. Feel free to contact us directly via email instead of on the forum. Even if you've written your Igor code as efficiently as possible, we can run it under a profiler and see if we might be able to improve Igor's performance.

Thanks for the info... this makes sense now.  We can live with 48 cores at a time :)