Wow guys I am impressed with some of the answers put up. Here is a summary...
What are the various ways that an object can be pinned?
People pretty much got this one right but they missed one.
GCHandles, Interop will "auto-magically" generate pins on the stac, the
"fixed" statement in C#, and Overlapped IO (There may be more here, in
particular I didn't have time to look into how reverse pinvoke was
working) but these are the main ones ... I like to think of it as 3
methods Stack (fixed, interop), GCHandle, and Overlapped.
I won't go too much into GCHandles ... they well work and are pretty
straight forward. If you are interested in how they work, reflector and
the SSCLI are a pretty good start. To get you started take a look at comdelegate.cs, take a look at InternalAlloc as a good starting point.
Fixed (which is most of the time how the marshaller works) is a bit
more interesting. It never creates a handle. If you look at reflector
you will see that the variable is created with pinned marked on it.
Basically the variable is defined as being pinned for the life of the
method, the GC recognizes by the call stack that the object is pinned.
You can see this with the !objsize command. From what I can tell in
Rotor it is not maintained in a table like gcinfo or exinfo but instead
is considerred pinned for the duration of the method. It is however set
to null when it is no longer in range so it can be collected. My guess
is that the commercial JIT works the same way.
Overlapped IO is a special type of Pin. If you start in
Overlapped.Pack and continue down the call chain you will eventually
end up in OverlappedData.AllocateNativeOverlapped.
[MethodImpl(MethodImplOptions.InternalCall)]
private extern unsafe NativeOverlapped* AllocateNativeOverlapped();
This is a "magic" method which allocates a NativeOverlapped struct from the CLR.
If we quickly jump into the SSCLI
we can find this demonic little internal call (yes I think 'internal
calls are evil') and in NativeOverlapped.cpp we will find...
if (userObject != NULL) {
if (overlapped->m_isArray == 1)
{
BASEARRAYREF asArray = (BASEARRAYREF) userObject;
OBJECTREF *pObj = (OBJECTREF*)(asArray->GetDataPtr());
DWORD num = asArray->GetNumComponents();
DWORD i;
for (i = 0; i < num; i ++)
{
GCHandleValidatePinnedObject(pObj[i]);
}
for (i = 0; i < num; i ++)
{
asArray = (BASEARRAYREF) userObject;
AddMTForPinHandle(pObj[i]);
}
}
else
{
GCHandleValidatePinnedObject(userObject);
AddMTForPinHandle(userObject);
}
}
So if m_IsArray is set it will treat all of the objects in m_userObject as an object[] otherwise it will treat it as an object ... but whatever is in there is implicitly treated as pinned.
When you further get into objecthandle.cpp you can really see just how much of a tax this type of pin is on the CLR code base ex:
if (!HndIsNullOrDestroyedHandle(pPinnedObj) && pPinnedObj->GetGCSafeMethodTable() == g_pOverlappedDataClass)
{
// reporting the pinned user objects
OverlappedDataObject *pOverlapped = (OverlappedDataObject *)pPinnedObj;
if (pOverlapped->m_userObject != NULL)
{
//callback(OBJECTREF_TO_UNCHECKED_OBJECTREF(pOverlapped->m_userObject), (ScanContext *)lp1, GC_CALL_PINNED);
if (pOverlapped->m_isArray)
{
pOverlapped->m_userObjectInternal = static_cast<void*>(OBJECTREFToObject(pOverlapped->m_userObject));
ArrayBase* pUserObject = (ArrayBase*)OBJECTREFToObject(pOverlapped->m_userObject);
Object **pObj = (Object**)pUserObject->GetDataPtr(TRUE);
DWORD num = pUserObject->GetNumComponents();
DWORD i;
for (i = 0; i < num; i ++)
{
callback(pObj[i], (ScanContext *)lp1, GC_CALL_PINNED);
}
}
else
{
callback(OBJECTREF_TO_UNCHECKED_OBJECTREF(pOverlapped->m_userObject), (ScanContext *)lp1, GC_CALL_PINNED);
}
}
if (pOverlapped->GetAppDomainId() != DefaultADID && pOverlapped->GetAppDomainIndex().m_dwIndex == DefaultADID)
{
OverlappedDataObject::MarkCleanupNeededFromGC();
}
}
Good for performance but yuck ...
What is heap fragmentation and why is it bad?
Heap fragmentation has been the subject of conversation here for
a little while. Yeah I know it was a it of a softball question but I
needed to have 5 questions and it came to mind :P A full answer can be
found here on Yun Jin's blog
When is heap fragmentation not important?
I think all of the answers here are valid. The one I was particularly looking for was ...
"When you are dealing with the LOH (Large Object Heap), its not compacted"
JD brings up another good point as the CLR is much better in 2.0 at
filling in gaps, and I think Craig brings up the best point of all
"I'd say heap fragmentation is important for some incredibly low number of Windows apps...like less than 5%.".
This is a very true statement.
The last two questions were really tricky
How can you tell if an object is pinned?
Tess aka Debugging
Goddess (who if you are reading this you should need no
introduction) on who's blog I usually find solutions to tricky SOS
issues like this says here...
If you are talking about managed heap fragmentation, there are two
kinds, 1) heap fragmentation caused by pinned objects, in which case
you can run !objsize (without parameters) to see what your pinned
objects are and where they are located.
This is really close to correct but not completely correct
.. !objsize will show you objects which are pinned through the
stack or gchandles but it won't show you those nasty little
overlapped pins. For those you just need to know that items in
m_userObject are pinned and hit them one by one :(
!GCHandles will also show you all of your GCHandles that are
out which should (except in the case of IO) show you what you are
interested in.
Can a pinned object be promoted?
It sure can! Rico M has a great explanation ... Steve also alluded to this.
Segment reuse is relatively straightforward. It’s to take
advantage of the existing gen2 segments that have a lot free space but
not yet empty (because if they were empty they would have been
deleted). Slide 27
demonstrates the problem without segment reuse. So before GC we are
running out of space in the ephemeral segment and we need to expand the
heap. We have 2 pinned objects. And I didn’t mark their generations
because it’s not significant to indicate their generations - they can
not be moved so they will stay in this segment and become gen2 objects
by definition. There might be pinning on the gen2 segments as well but
since that doesn’t affect illustrating the point I preferred to keep
the picture simple and left them out.
So after GC we allocate a new segment. The old gen1 in the old
ephemeral segment was promoted to gen2 and the old gen0 now lives in
the new ephemeral segment as the new gen1 and we will start allocating
after gen1 on this segment. The pinned objects are left in the old
ephemeral seg since they can not be moved. They are part of gen2 now
because they live in a gen2 segment.
You might be asking yourself how on earth I came accoss these
Overlapped pins, well in working with the BufferManager in previous
posts I ran SOS alot to look at my heap fragmentation. I saw something
quite odd; my buffer manager version had the exact same pins reported
in !GCHandles (OverlappedDatas) as the non-bufferred version yet it had
way less heap fragmentation. The pins on my actual buffers were never
being shown so in trying to track them down to prove it was doing what
I claimed it did I came accross all of this ... what a night that
was! Nothing better than a good bottle of Sancerre and SOS outputs that
make no sense (sometimes the Sancerre helps:))
So what to do? Well for the pinned objects in the debugger I
talked with Steve about adding support to his excellent SOSEX tool for
this.