Subscribe: /* Rambling comments... */
http://www.lenholgate.com/rss2.xml
Added By: Feedage Forager Feedage Grade A rated
Language: English
Tags:
buffer  bug  code  framework  jetbytetools  new  release  server framework  server  socket  studio  visual studio  visual 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: /* Rambling comments... */

www.lenholgate.com - Rambling Comments - Len Holgate's C++ progamming blog



/* Rambling comments... */ Len Holgate's thoughts on this and that... Mainly test driven software development in C++ on Windows platforms...



Updated: 2016-07-11T16:04:10Z

 



Supporting Visual Studio 2015 Update 3 - AsynchronousEvents

2016-07-11T16:04:10Z

Visual Studio Update 3 adds some new warnings for precompiled header generation. One of these, C4598, will prevent precompiled header builds of framework code due to our warnings as errors policy. A fix for this issue is to add the...

Visual Studio Update 3 adds some new warnings for precompiled header generation. One of these, C4598, will prevent precompiled header builds of framework code due to our warnings as errors policy.

A fix for this issue is to add the new warning to the list of disabled warnings in the project file. The easiest way to do this is to do a search and replace across your source tree for *.vcxproj files and replace "4627;4654;" with "4627;4654;4598;".

I need to work out what the implications of this compiler change are on how our precompiled headers are generated.

(image)



Latest release of The Server Framework: 6.7 - AsynchronousEvents

2016-06-25T16:04:45Z

Version 6.7 of The Server Framework was released today. This release is mainly a code simplification release which removes support for legacy compilers and operating systems. See here for more details. However, there are some breaking changes where smart... Version 6.7 of The Server Framework was released today. This release is mainly a code simplification release which removes support for legacy compilers and operating systems. See here for more details. However, there are some breaking changes where smart buffers have replaced buffer references and this causes function signature changes. In addition there has been considerable churn in the Streaming Media Option Pack with knock on changes in the HTTP library code which needed to be made more complete to deal with serving HLS streams. As always, see the release notes here, for full details of all changes. Breaking changes: Breaking change JetByteTools::IO::CAsyncFileReader, JetByteTools::IO:: CAsyncFileWriter, JetByteTools::IO::CAsyncFileWriterEx and JetByteTools::IO::IAsyncIOStream now work in terms of JetByteTools::IO:CSmartBuffer where possible. This potentially reduces the reference counting activity on the buffers. Breaking change Where possible JetByteTools::IO::CSmartBuffer has replaced raw pointers and references to buffers. This massively reduces the need to reference count buffers during normal I/O operations and increases performance, especially on NUMA architectures. Bug fixes: Bug fixes to JetByteTools::IO::CBuffer around the usage of m_maxBytesToRead. Bug fix to JetByteTools::Win32::CCallbackTimerQueueEx::BeginTimeoutHandling() to prevent incrementing m_nextTimeoutHanlde causing the value to wrap to InvalidTimeoutHandleValue which was possible, but unlikely. Changes: Dropped support for Visual Studio 2005 and Visual Studio 2008. Dropped support for Windows XP. Removed JETBYTE_HAS_INTERLOCKED_64 as it's true on all supported platforms now. Removed JETBYTE_WARN_ON_SOCKADDR_STORAGE_DEF and JETBYTE_WARN_ON_WSPIAPI_COUNTOF_DEF as these are no longer relevant. Removed JETBYTE_PERF_STREAM_SOCKETS_SKIP_EVENT_ON_HANDLE, JETBYTE_PERF_DATAGRAM_SOCKETS_SKIP_EVENT_ON_HANDLE and JETBYTE_PERF_FILE_WRITER_SKIP_EVENT_ON_HANDLE as these are always enabled now. Removed JETBYTE_PERF_STREAM_SOCKETS_SKIP_MARSHAL_TO_IO_POOL and JETBYTE_PERF_DATAGRAM_SOCKETS_SKIP_MARSHAL_TO_IO_POOL as these are always enabled now. Marshalling was only required to work around Windows XP's I/O cancellation on thread termination policy. Removed JETBYTE_DEPRECATE_SHARED_CRITICAL_SECTIONS, JETBYTE_DEPRECATE_SHARED_LOCK_SOCKETS, JETBYTE_DEPRECATE_CRITICAL_SECTION_2, JETBYTE_DEPRECATE_LOW_CONTENTION_BUFFER_ALLOCATOR, and JETBYTE_DEPRECATE_TLS_BUFFER_ALLOCATOR as these features are no longer available. Removed SecureCRT.h as it's no longer required. It was used to map differences between the Visual Studio 2005 CRT and the "new" secure CRT. Added the macro, SuppressLNK4221Warning(), which can be put into a file in order suppress the MS Visual C++ Linker warning 4221 - "warning LNK4221: no public symbols found; archive member will be inaccessible" Turn off JETBYTE_INTRUSIVE_RED_BLACK_TREE_INTERNAL_STATE_FAILURE_EXCEPTIONS by default. Removed JetByteTools::IO::CLowContentionBufferAllocator and JetByteTools::IO::CTLSBufferAllocator. Added JetByteTools::IO::CBufferBasedBufferAlloctor which is a simple shim to allow an instance of JetByteTools::IO::IBuffer to be used as an implementation of JetByteTools::IO::IAllocateBuffers. Removed JetByteTools::IO::IIOPool::DispatchToAll() as it was only required for issuing CancelIO() calls to all I/O threads on XP where CancelIOEx() wasn't available. Added an overload of JetByteTools::IO::CNonPooledBuffer::Create() that takes a bufferSize and a pointer to data and a data length so that you can create a buffer that initially contains some data but that is larger than that data. Added lots of standard buffer functionality to JetByteTools::IO::CNonPooledBuffer before deprecating it in favour of normal buffers obtain[...]



Visual Studio Component Cache bug looks like it's here to stay... - Rambling Comments

2016-06-08T11:11:07Z

So, it's nearly a year since I first started noticing issues with VS2015 on my build servers. The bug in question now has an entry on Microsoft Connect and Google can help with some work arounds which don't require turning...

So, it's nearly a year since I first started noticing issues with VS2015 on my build servers. The bug in question now has an entry on Microsoft Connect and Google can help with some work arounds which don't require turning the machine off and on again a random number of times until it works... There's even a Visual Studio extension that fixes the issue for you (!).

I find it disappointing that this hasn't been fixed, it's a fundamental usability issue which seems to be causing lots of people lots of pain. It's probably not too bad if you're running Visual Studio as a developer in the 'normal' way; especially if the extension can fix the issue for you when it happens, but on a build machine it's a pain. Of course, it only ever happens just after you kick off a build and leave the office. If you're sitting there waiting for it to happen the problem never seems to manifest...

(image)



6.7 - Potentially faster code, in some circumstances... - AsynchronousEvents

2016-06-25T10:58:17Z

I hinted at the end of the last post that the 6.7 release might increase performance a little. Well, whilst the bulk of the changes in 6.7 are purely code cleaning and the removal of legacy support there is a...

I hinted at the end of the last post that the 6.7 release might increase performance a little. Well, whilst the bulk of the changes in 6.7 are purely code cleaning and the removal of legacy support there is a fairly major functional change as well.

In most situations references or pointers to I/O buffers have been replaced with smart pointers. This change may cause some issues during an upgrade as you need to change some function signatures from IBuffer refs to CSmartBuffers. The advantage is that in many servers there will no longer be any need for buffer reference counting during normal I/O operations.

The Server Framework relies on reference counting to keep the objects that are used during the asynchronous operations alive until those operations complete. So we increment a counter on the socket object and also on the buffer object when we initiate an operation and then decrement the counters when the operation completes. I'm sure there are other ways to manage the object lifetime but this has worked well for us.

The problem is that these increments, although they look like cheap operations, can be quite expensive, especially on NUMA hardware.

Whilst there's not much we can do about the reference count on the socket object, the buffer doesn't really need to be reference counted most of the time. Or, more's the point. The initial reference can (and should) be passed along rather than each stage taking and releasing its own reference. With a buffer you generally only want to be accessing it from one thread at a time and so you allocate it and then issue an operation and pass the reference you have off to the operation. When the operation completes the code captures the reference and takes ownership of it and then the buffer can be processed. If you're lucky you can then use the same buffer for a response to the operation and pass it back to the framework again.

This requires a few changes to your code but it's fairly straight forward. Your OnReadCompleted() handler will give you a CSmartBuffer and if you want to retain ownership of it after the handler returns then you simply need to detach the buffer from the CSmartBuffer you were given.

This is only "potentially faster" as it really depends on the structure of your server and how you deal with our I/O buffers but the framework is no longer standing in the way of this kind of optimisation, and we've removed a couple of reference adjustments in the normal operation flow.

(image)



Another release is coming... - AsynchronousEvents

2016-06-25T16:06:14Z

We've only just shipped Release 6.6.5 of The Server Framework but we already have another release that's just about to ship. This isn't because some horrible bug has slipped through our testing, it's because we've been planning to produce a... We've only just shipped Release 6.6.5 of The Server Framework but we already have another release that's just about to ship. This isn't because some horrible bug has slipped through our testing, it's because we've been planning to produce a 'clean up' release for some time. 6.7 is that release. Lets be straight here, 6.7 is a release for us more than for you. The aim is to simplify our build/test and release process, remove dead code whilst introducing no new bugs and removing no functionality that you rely on. So what does 6.7 achieve. Well, for a start we drop support for Visual Studio 2005 and 2008 and also for Windows XP. Removing support for these legacy compilers and operating systems means that we can remove all the code that was required just to support them. This massively simplifies our code base without removing anything that the code actually relies on to run on modern operating systems. Windows Vista introduced massively important changes to asynchronous I/O and we have supported these changes for a long time (over 8 years!). The code required to jump through hoops to make code running on Windows XP behave was complex. For example, Windows XP would cancel outstanding I/O requests if the thread that issued them exited before the I/O request completed. We had a marshalling system in place to ensure that I/O operations were only ever executed on threads that we controlled so that you'd never be faced with unexpectedly cancelled operations. All of that can go now. Removing XP also means we no longer need to maintain an XP machine in our build farm. It's one less configuration that needs to be built and tested before a release. Dropping support for VS2005 and 2008 removes 4 complete sets of builds (x86 and x64 for each compiler) plus all of the conditional code that was required to support the older compilers. At last we can start moving towards a slightly more modern C++, perhaps. Some old code has been removed; there's no need, on modern operating systems, to share locks. This worked really well back in the day, but, well, we were running on Windows NT at the time and resources were much more limited than they are now. All of the "Shared Critical Section" code is now gone. This has knock on effects into the Socket Tools library where all of the shared lock socket code has been removed. Nobody should be using that in 2016 anyway! You can no longer set a critical section's spin count in the socket allocator, it never really worked anyway as the lock was used for too many different things. Some experimental code has also been removed; The TLS and Low Contention buffer allocators are gone. The horrible "dispatch to all threads" cludge has been removed from the I/O pools (it was only there to support pre-Vista CancelIO() calls which are no longer needed now that we have CancelIOEx()). The original callback timer queue that was based on GetTickCount() and which spawned Len's "Practical testing" series of blog posts (back in 2004!) has gone. There's no need for the complexity when all supported operating systems have GetTickCount64(). Finally we've slimmed down our set of example servers. Removing servers which didn't add much value or which duplicated other examples. Again, this speeds our release process by speeding up the build and test stage as there are fewer servers to build and fewer tests to run. So, what's in it for you? Well, a faster build/test/release cycle so new functionality and bug fixes can be released quicker and potentially faster code in some circumstances. There's no great rush to upgrade if you don't want to, but we'll be focusing on the 6.7 code base going forwards. [...]



Latest release of The Server Framework: 6.6.5 - AsynchronousEvents

2016-06-25T17:27:34Z

Version 6.6.5 of The Server Framework was released today. This release is mainly a feature release with a few bug fixes. As always, see the release notes here, for full details of all changes. Bug fixes: Bug fix to... Version 6.6.5 of The Server Framework was released today. This release is mainly a feature release with a few bug fixes. As always, see the release notes here, for full details of all changes. Bug fixes: Bug fix to JetByteTools::Socket::TAsyncSocket::ProcessAndGetNextOperation(). We now wrap the body of the function in an exception handler and abort the connection with JetByteTools::Socket::ConnectionClosureReason::FatalErrorAbort if an exception is thrown during processing. This fixes a bug whereby the connection would otherwise hang in these situations. Bug fix to JetByteTools::Win32::TReentrantLockableObjectTracksLockingThread to add a cast which is needed for some compilers. Bug fix to JetByteTools::WebSocket::HyBi::CProtocolHandler::HandleData() to remove an incorrect internal state validation exception which would generate spurious "No space left in read buffer." exceptions. Changes: Added JetByteTools::IO::IBuffer::OnBufferAddedToPool(), JetByteTools::IO::IBuffer::OnBufferRemovedFromPool(), JetByteTools::IO::IManageBufferLifeCycle::OnBufferAddedToPool() and JetByteTools::IO::IManageBufferLifeCycle::OnBufferRemovedFromPool(). These allow for correct management of dynamically allocated buffer data. Previously JetByteTools::IO::IMonitorBufferAllocation::OnBufferAllocated() was called whenever a buffer was allocated from the allocator OR a custom pool and JetByteTools::IO::IMonitorBufferAllocation::OnBufferReleased() was only called when the buffer was released to the allocator. This made it impossible to manage dynamically allocated buffer data that was created and destroyed by a derived allocator object using the monitoring interface to monitor the underlying allocator. Now JetByteTools::IO::IMonitorBufferAllocation::OnBufferReleased() is called when the buffer is added to a custom pool and this allows the monitor to match allocations and releases exactly. Added JetByteTools::IO::IAllocateBuffers::AllocateCustomSizedBuffer() which allows you to allocate a custom buffer from any allocator. The buffer will have the same user data slots as any other buffer allocated from the allocator but if it is larger than the allocator's buffer size the new buffer will NOT be pooled upon release. Added JETBYTE_ILLEGAL_BUFFER_USER_DATA_EXCEPTIONS which defaults to 0 and when set turns on range checking for user data indices in JetByteTools::IO::CBuffer and an index out of range exception is thrown if necessary. Added the ability to pass JUST a custom allocator into the constructor of JetByteTools::IO::CRotatingAsyncFileLog and allow the file log to manage the other resources that it needs directly. Previously you had to provide all of the resources or none. Added an override for JetByteTools::Socket::ISocketCallback::OnError() which takes a DWORD error code so that client code can selectively ignore errors by error code. Previously the only way to ignore errors was by the error message itself which is localised and therefore impossible to match reliably. Rationalised the status changes for JetByteTools::Socket::TStreamSocketServer<> and JetByteTools::Socket::TStreamSocketServerEx<> to remove some strangeness when shutdowns are initiated after the server has already shut down. Added new value to JetByteTools::Socket::ConnectionClosureReason, FatalErrorAbort. This is used if the framework itself needs to abort a connection for any reason. Added JetByteTools::Socket::CStreamSocketNamedConnectionCollection::GetConnectionName() which returns the name of a given connection. Added JetByteTools::Socket::CStreamSocketBroadcastableConnectionCollection::BroadcastToAllExcept() which broadcasts a buffer to all connections except the supplied connection. Added [...]



Things have been busy! - Company News

2016-05-24T14:22:46Z

We've been busy with lots of custom development work over the last few months. Working with Eonic Gaming on their server for the Turf Battles Triumphus 3D MMORPG. Working with one of our security company clients on their video...
We've been busy with lots of custom development work over the last few months. Working with Eonic Gaming on their server for the Turf Battles Triumphus 3D MMORPG. Working with one of our security company clients on their video streaming server, adding RTP over TCP and HLS streams to the RTP over UDP of the original design. Improving the performance and functionality of the pluggable server that we wrote for one of our other gaming clients as they scale out into the cloud. And doing some consultancy for existing clients who want to expand their understanding and usage of The Server Framework.

The video streaming work was fun and went well and it was nice to have a well specified project using internet standards that we could really get our teeth into.

The work for Eonic Gaming has been especially interesting as we've worked closely with them to address their initial scalability issues and switch from a single server solution to a more traditional MMORPG style multiple server design.

We've also been working on several new releases of The Server Framework, one in the 6.6.x tree, a new, slimmed down 6.7 and some investigation into what could become massively modernised 7.0 release. But we'll see...

Now that things are finally becoming a little bit calmer our focus is on getting the 6.6.5 and 6.7 releases finalised and out the door.
(image)



TLS 1.2 handshake failure for certificates signed with MD5 - AsynchronousEvents

2016-06-25T10:59:19Z

A while back a client of mine had an issue with a TLS 1.2 connection failing during the TLS handshake. We couldn't see any issues with the code and if we only enabled TLS 1.1 on the server then the... A while back a client of mine had an issue with a TLS 1.2 connection failing during the TLS handshake. We couldn't see any issues with the code and if we only enabled TLS 1.1 on the server then the connection handshake worked just fine. Eventually we tracked the issue down to the fact that the certificate in use had been signed with MD5 and that MD5 isn't a valid hash algorithm for TLS 1.2 and so the handshake failed when SChannel attempted to validate the certificate and found that it was unacceptable. There's a Microsoft blog posting about this problem here. Annoyingly the error message for this is: "The client and server cannot communicate, because they do not possess a common algorithm." which is bang on the money, but doesn't help much until after you've worked out what the problem is. We were already dumping out the enabled algorithms and protocols and couldn't understand why the connection either wasn't working or wasn't downgrading to TLS 1.1. I've now added some certificate investigation code to the point where my example SChannel servers set up their contexts. This at least allows me to warn and degrade to TLS 1.1 if the certificate is not going to work with TLS 1.2. In production systems I expect we'd just fail to start up. CCredentials::Data data; data.cCreds = 1; data.paCred = &pCertContext; // Note that for TLS 1.2 you need a certificate that is signed with SHA1 and NOT MD5. // If you have an MD5 certificate then TLS 1.2 will not connect and will NOT downgrade // to TLS 1.1 or something else... DWORD dataSize = 0; if (::CertGetCertificateContextProperty( pCertContext, CERT_SIGN_HASH_CNG_ALG_PROP_ID, 0, &dataSize)) { ByteBuffer buffer(dataSize); BYTE *pData = buffer.GetBuffer(); if (::CertGetCertificateContextProperty( pCertContext, CERT_SIGN_HASH_CNG_ALG_PROP_ID, pData, &dataSize)) { const _tstring signHashDetails(reinterpret_cast(pData), dataSize); if (signHashDetails.find(_T("MD5")) != _tstring::npos) { if (enabledProtocols == 0 || enabledProtocols & SP_PROT_TLS1_2) { OutputEx(_T("Certificate is signed with: ") + signHashDetails); OutputEx(_T("MD5 hashes do not work with TLS 1.2 and connection downgrade does NOT occur")); OutputEx(_T("This results in failure to connect. Disabling TLS 1.2")); if (enabledProtocols) { enabledProtocols &= ~SP_PROT_TLS1_2; } if (!enabledProtocols) { // if enabledProtocols is zero then all protocols are // enabled, so we need to explicitly enable everything // except TLS1.2 enabledProtocols = SP_PROT_SSL3TLS1; } } } } } data.grbitEnabledProtocols = enabledProtocols; [...]



Latest release of The Server Framework: 6.6.4 - AsynchronousEvents

2016-06-25T17:28:47Z

Version 6.6.4 of The Server Framework was released today. This release is mainly a bug fix release for clients using WebSockets over SSL. As always, see the release notes here, for full details of all changes. Bug fixes: Bug... Version 6.6.4 of The Server Framework was released today. This release is mainly a bug fix release for clients using WebSockets over SSL. As always, see the release notes here, for full details of all changes. Bug fixes: Bug fix to JetByteTools::Win32::CallbackTimerQueueBase which prevents the timeout handle from ever being incremented to zero. It's unlikely but possible. Bug fix to JetByteTools::Win32::CBuffer and JetByteTools::Win32::CLockFreeBuffer to make code take notice of when m_maxBytesToRead is set. This was causing issues with WebSockets over SSL where the WebSocket code was setting a max read value and the SSL connector was ignoring it and providing more data than it should. Bug fix to JetByteTools::SSPI::SChannel::CAsyncConnector::PerformWrite() so that we deal with exceptions appropriately. Bug fix to JetByteTools::Socket::CFilterDataBase::WriteData() so that we deal with exceptions appropriately. Bug fix for a highly unlikely possible situation in JetByteTools::WebSocket::HyBi::CProtocolHandler::HandleData() where a buffer in the read request buffers queue contains no space for more data. Changes: Adjusted all includes to remove relative paths and instead be absolute in relation to the library root. So, rather than #include "..\Blah.h" it would always be #include "JetByteTools\Lib\Blah.h" this is due to a new warning in Visual Studio 2015 update 1. Added JetByteTools::Socket::ListenBacklogMaxwhich is a JetByteTools::Socket::ListenBacklog value that can be used to cause the listen queue for a socket to be set to SOMAXCONN. Added JetByteTools::Socket::TStreamSocketServerEx::SoMaxNumPendingAccepts which is the number of accepts that are posted if a listen backlog of JetByteTools::Socket::ListenBacklogMax is specified for the number of pending accepts. At present this is set to 200. Added JetByteTools::Socket::TStreamSocketServerEx::SetListeningParameters() which can be called after construction but before starting to accept connections and can specify the listen backlog and number of receives to post separately. This allows you to set the listen backlog to SOMAXCONN using JetByteTools::Socket::ListenBacklogMax and set the number of receives to something other than JetByteTools::Socket::TStreamSocketServerEx::SoMaxNumPendingAccepts. You can also set a flag which tells the server not to close the listening socket when the server reaches a connection limit, instead it will simply stop posting accepts and so new connections will get to wait in the listen queue until they time out or until the server posts a new accept request. Bug fix to to JetByteTools::Socket::TStreamSocketServerEx so that OnMaxConnections() is called correctly when the connection limit is reached. Previously it would often only be called the first time that the limit was reached. [...]



SChannel ALPN support, documentation "bug"... - Rambling Comments

2015-11-05T15:27:37Z

I'm looking at adding support for the TLS Application-Layer Protocol Negotiation Extension to The Server Framework as a precursor to looking into HTTP 2.0 support. This TLS extension is supported by both SChannel and OpenSSL. It's relatively new (only having been...

I'm looking at adding support for the TLS Application-Layer Protocol Negotiation Extension to The Server Framework as a precursor to looking into HTTP 2.0 support. This TLS extension is supported by both SChannel and OpenSSL. It's relatively new (only having been included in SChannel since June 2014 (!)) but there seems to be no documentation on MSDN for the structures and how to use them to add the appropriate entries to the TLS handshake and then to select and validate the protocol of your choice... Searching today with Google for something like "SEC_APPLICATION_PROTOCOL_NEGOTIATION_EXT" just gives just the usual selection of hits to people who think it's clever to run the Windows header files through their document generators and then publish the results on the web...

It's slightly easier with OpenSSL, but only because I can look at the source...

Anyway, long story short, I have the OpenSSL version working and am getting an HTTP 2.0 Connection Prefix from Chrome... I guess, for now, SChannel will have to wait.

I expect I'm being dense...

(image)



GetVersionInfoEx() deliberately broken in Windows 8.1/10 - Rambling Comments

2015-09-16T10:13:14Z

There should be a Developer Advocate position within Microsoft. They seriously need someone who doesn't "Drink the Kool-Aid" who can step in and tell the Program Managers or Marketing bods or whoever it is who makes all the really STUPID...

There should be a Developer Advocate position within Microsoft. They seriously need someone who doesn't "Drink the Kool-Aid" who can step in and tell the Program Managers or Marketing bods or whoever it is who makes all the really STUPID decisions that the decision is stupid and that they should get back in their box and leave the developers who support the Windows ecosystem by building software that runs on it to just get on with their lives...

Yes, this is going to be a rant...

So someone has decided that being able to query which version of the Operating System that your program is running on is a bad thing. If you try and call GetWindowsVersionEx() on Windows 8.1 or Windows 10 and you haven't specifically set your application's manifest to say that it's compatible with Windows 8.1 or Windows 10 then GetWindowsVersionEx() will lie to you and tell you that you're running on Windows 8.

Excuse me, but that's just brain dead. There's NO valid reason for doing this because you can't know WHY the caller needs this information.

OK, so it might be better, in some circumstances, to be checking for the presence of various function entry points in various dlls and dynamically working out that what you need is present BUT that's not the only reason that you might want to know what OS you're running on and it's not even the only GOOD reason.

My current situation is that Windows 10 spawns a conhost.exe process whenever you start a console process. The test code that I have for my Win32 Job API wrapper includes some checks to determine that the correct job accounting data is being returned from some test programs that it spawns. This accounting data is different under Windows 10 as there are more processes running within the job. Ideally I'd like to acknowledge that this is an OS difference, adjust the test to know that when it runs on this OS it needs to expect slightly different results and then move on with my life. Of course, as Microsoft has deliberately chosen to break the API that would let me cleanly work out what version I'm running on I have to spend time writing ranty blog postings and crufting up crappy work arounds.

(image)



Visual Studio 2015 Bugs - Save As existing file crash - Rambling Comments

2015-08-28T14:45:31Z

As I said a while ago, Visual Studio 2015 appears to have lots of pretty serious bugs. One that probably bites me more than many people is this Save As bug. If you have a project and you select a...

As I said a while ago, Visual Studio 2015 appears to have lots of pretty serious bugs.

One that probably bites me more than many people is this Save As bug. If you have a project and you select a file in that project and do a "Save As" such that the target file is also in the project then the VS2015 IDE crashes.

I tweeted about this a while back, and the Visual Studio Program Manager, Cathy Sullivan, wanted me to create a new project and see if the problem happened with it... Not quite sure why she couldn't do this herself, but anyway, here's a zip of a project created with VS2015 where there's a file called SaveMeAsTargetver.h. If you save that file as targetver.h then the IDE crashes.

(image)



Latest release of The Server Framework: 6.6.3 - AsynchronousEvents

2015-08-25T09:38:08Z

Version 6.6.3 of The Server Framework was released today. This release is mainly a bug fix release but it also adds support for Visual Studio 2015 and Windows 10 (though we don't explicitly use any Windows 10 APIs). There... Version 6.6.3 of The Server Framework was released today. This release is mainly a bug fix release but it also adds support for Visual Studio 2015 and Windows 10 (though we don't explicitly use any Windows 10 APIs). There are quite a lot of small changes due to us running Gimpel Lint over the code. Most of the changes will have fixed potential issues rather than issues that have actually been reported. However, there are two serious fixes for the SChannel Option Pack for an issue that was introduced in Release 6.6.2 and which could cause corruption of the data flow in some situations and stalling and CPU pegging in another. There is also a fix which may prevent UDP datagram corruption, See here for more details. As always, see the release notes here, for full details of all changes. All clients using Release 6.6.2 of the SChannel Option Pack are advised to upgrade to this release. Bug fixes: JetByteTools::IO::CBuffer::Clear() which now correctly sets m_ioSize to zero. JetByteTools::IO::CBufferList::BufferData so that it cannot access pBuffer before it's initialised. JetByteTools::IO::CLockableBufferProcessor::~CLockableBufferProcessor() to remove potential buffer leak. JetByteTools::IO::CAsyncFileReader where we were checking the validity of data response flags incorrectly. JetByteTools::SSPI::SChannel::CAsyncConnector::PerformWrite() where we could sometimes send invalid data. JetByteTools::SSPI::SChannel::CAsyncConnector where we could sometimes get stuck in an infinite loop if we had decrypted data available but no application level read pending. JetByteTools::Service::CService::ServiceMain() so that we set and clear the m_serviceStarting flag correctly. Fix to all WSASend() and WSARecv() calls to prevent multiple threads from calling into the API on the same socket at the same time. See here for more details. JetByteTools::Win32::CCallbackTimerWheel::EndTimeoutHandling() where we were leaking one shot timers. JetByteTools::Win32::CThreadPool::OnThreadPoolThreadStopped() and JetByteTools::Win32::CThreadPoolEx::OnThreadPoolThreadStopped() so that the call to the monitor's OnThreadPoolStopped() function is consistent and in the right place. JetByteTools::Win32::CSmartHandle to deal with closing pseudo thread or process handles. Fix to JetByteTools::Socket::CFilterDataBase to ensure that each filter knows how many writes it has issued and that they pass completions to the next layer when they should. JetByteTools::Win32::TReusableIdManager::InternalTryFree() so that we now correctly merge intervals. Note that this bug was purely in the way we stored the intervals (which was less efficient than it could be) and did not affect the function of the manager. JetByteTools::Win32::TZeroInitialiseExpandableBuffer::Resize() so that it doesn't trash memory if you reduce the size of the buffer. JetByteTools::Win32::TZeroInitialiseExpandableBuffer a fix to the assignment operator so that it actually compiles and works. Changes: Ran Visual Lint using Gimpel PC Lint on all code and adjusted to remove warnings and fix bugs. Added support for Visual Studio 2015. Removed all use of exception specifications. We only ever used throw() but that's now gone too. Protected non-virtual destructors on interfaces are now virtual even though they you can't delete the object via the interface. All destructors that could throw exceptions now have optional "log and swallow" exception handlers which are enabled by default. This is better than ignoring the problem and being face[...]



Visual Studio 2015 RTM - I hope the first service release is soon... - Rambling Comments

2015-07-24T20:19:26Z

As usual I have used VMs to play with the CTP releases for the new version of Visual Studio and fixed up new warnings and build failures in my source so that I'm not too surprised when the RTM comes...

As usual I have used VMs to play with the CTP releases for the new version of Visual Studio and fixed up new warnings and build failures in my source so that I'm not too surprised when the RTM comes along.

Unfortunately as soon as I put VS2015 RTM on my build server and started running it as part of my CI system I started noticing strange things...

The first is that it seems to be possible to get VS2015 into a state where it will refuse to load C++ projects. I've no idea how I've managed this, but it's now happened twice. Rebooting the machine seems to fix things. The log file that the failure indicates might help shows that it's trying to load a the "Visual Studio C++ Project System Package" and it's getting an E_POINTER result...

The second issue seems to be a race condition in shutting down the new process that's used to help various parts of VS communicate with each other, VSHub.exe. My CI system will fire up a Visual Studio instance to build a project using /build and once that completes another project is likely to get built, sometimes two VS2015 builds will follow one another, other times there are other versions of Visual Studio building in between. VS 2015 appears to start an instance of VSHub.exe but only if one isn't running (or, more likely, it always starts one but if one's already running then the new ones shut down). Then when the last instance of VS quits VSHub.exe hangs around for a moment and then shuts down... It seems to crash if it's shutting down when a new instance of VS2015 is started (and, presumably decides that the VSHub.exe that is shutting down is the one to use). A fix for this issue is simply to keep VSHub.exe alive for the entire time that all VS2015 builds need to run and I can do this by having an instance of VS2015 open on the build machine whilst the CI system is running...

Thanks to Abe10, in this thread, for the workaround for the VSHub.exe issue.

6 out of 10 for this VS release Microsoft, try harder. I hope the Windows 10 team have better QA.

(image)



The real solution to "error MSB4175: The task factory "CodeTaskFactory" could not be loaded from the assembly" - Rambling Comments

2015-07-15T07:55:56Z

I now have a real solution to the problem that I outlined on Saturday night and this solution, unlike the one developed during my "Macallan driven development" session on Saturday evening actually works. The problem is that when using code...

I now have a real solution to the problem that I outlined on Saturday night and this solution, unlike the one developed during my "Macallan driven development" session on Saturday evening actually works.

The problem is that when using code to run build using VS2010 by running a project into devenv.com I get the following error message:

C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\CodeAnalysis\Microsoft.CodeAnalysis.targets(214,5): error MSB4175: The task factory "CodeTaskFactory" could not be loaded from the assembly "C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Build.Tasks.v12.0.dll". Could not load file or assembly 'file:///C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Build.Tasks.v12.0.dll' or one of its dependencies. The system cannot find the file specified.

The project builds correctly from the command window using the exact same command line as I use in my calls to CreateProcess().

I was on the right track on Saturday evening. It is an environment issue, just not the one that I thought it was. I also now understand why I thought I'd solved the problem on Saturday night only to discover that the fix didn't work when I checked it again on Sunday morning.

The problem is caused by the presence of "VisualStudioVersion" in the environment. This variable is set to 12.0 when my task runner program is run from within VS2013 either when debugging or running without the debugger attached but launching from within VS2013. The presence of this variable in the task runner's environment somehow causes VS2010 to use the latest version of MSBuild that ships with VS2013. When doing this MSBuild gets confused about which .Net framework directory to load assemblies from and fails to locate its dll. Removing the "VisualStudioVersion" from the inherited environment before launching VS2010 programatically fixes the problem.

Running my task running from a normal command prompt, rather than from inside of VS2013, allows it to work correctly without needing to remove the variable (as the variable is only added by VS2013 itself). I expect that I tested like this on Saturday night and that's what made me think I'd solved the problem them.

VS2013 actually adds several "VisualStudio" variables to the environment and I now remove all of them before running any task from my task runner.

(image)



Solution to "error MSB4175: The task factory "CodeTaskFactory" could not be loaded from the assembly" - Rambling Comments

2015-07-12T07:47:58Z

Updated: 12 July 2015 - It seems that this is all incorrect... Upon running my tests again this morning I find that the x64 task runner also fails to run VS2010 correctly and so the environment differences are unlikely to...

Updated: 12 July 2015 - It seems that this is all incorrect... Upon running my tests again this morning I find that the x64 task runner also fails to run VS2010 correctly and so the environment differences are unlikely to be the cause...

So, having decided that my continuous integration system could be better, and having looked at JetBrains' TeamCity and decided that it doesn't quite fit I'm looking back at some code that I wrote back in 2008 when I last thought about this kind of thing...

I have some code which works with trees of dependant tasks and triggers sub tasks once the tasks they depend on complete successfully. The code uses CreateProcess() and Win32 Jobs to control the tasks and it can run builds and tests using the various flavours of Visual Studio installed on the box in question.

So far, so good as a starting point for an agent that can run multiple builds on a machine...

My problem, well, one of them, being that when using VS2010 by running a project into devenv.com I get the following error message:

C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\CodeAnalysis\Microsoft.CodeAnalysis.targets(214,5): error MSB4175: The task factory "CodeTaskFactory" could not be loaded from the assembly "C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Build.Tasks.v12.0.dll". Could not load file or assembly 'file:///C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Build.Tasks.v12.0.dll' or one of its dependencies. The system cannot find the file specified.

This is annoying, to say the least, VS2005, 2008 and 2012 and 2013 all work fine, it's just 2010 that fails. I googled the issue and the general consensus is that once you've installed VS2013 a different version of MSBuild is used. Of course that doesn't answer the question of why it 'works on my machine' except when my task runner is running the command...

Eventually I got the task runner to dump the environment and did the same from the command prompt where the command worked. The difference being that the command prompt that worked had an 'x64' environment and the task runner had a different, 'x86/win32' environment... This was because the task runner was a Win32 process. Running the task runner as x64 fixed the issue. So there's probably a bug in the VS2013 installation code that fails to set some environment variables for the Win32 command prompt that are required once it's "screwed with" the existing MSBuild installation on the box...

I'm expecting that manually adding the missing bits to the 'x86' environment will mean that the task runner will work as a Win32 build AND an x64 build...

(image)



Bug hunting - Rambling Comments

2015-07-01T10:14:53Z

I've just spent a day tracking down a bug in a pending release of The Server Framework. It was an interesting, and actually quite enjoyable, journey but one that I shouldn't have had to make. The bug was due to... I've just spent a day tracking down a bug in a pending release of The Server Framework. It was an interesting, and actually quite enjoyable, journey but one that I shouldn't have had to make. The bug was due to a Windows API call being inserted between the previous API call and the call to GetLastError() to retrieve the error code on failure. The new API call overwrote the previous error value and this confused the error handling code for the previous API call. This was a bit of a "school boy error" and it was made worse by the fact that the code in question had a comment which clearly explained why nothing should be placed between the API call and the call to GetLastError(). So, what went wrong? This bug has severely delayed the release of 6.6.3 of The Server Framework. This is mainly due to the fact that it caused many of my "black-box" server tests to fail spectacularly with all manner of symptoms and with crash dumps indicating invalid memory accesses. This, in turn, caused several days of head scratching which used up the time I'd allocated to push the release out. Then client work took over and the release had to wait as I had no time slots large enough to allocate to finding and fixing the bug. Annoyingly all of the unit tests passed. It was only under stress that the code in question failed because the code path in question wasn't exercised in the unit tests. This is why I have the "black-box" server tests as part of my build and release process but they don't get run as often as they should as I don't practice continuous integration for various reasons. CI would have spotted the problem as soon as it was introduced into the codebase and it would have been easy to triage the changes, locate and fix the issue. The release in question has always been a collection of unimportant minor changes that has been allowed to accumulate issues over time and that had no real plan to it. All of the changes were being merged in from client fixes and so all of the fixes were active in the field to various degrees. It should have been easy to push the 'build, test and release' button at any time and push the release out. However when I did this the lack of CI meant that this was the first time that the code had been stress tested and, obviously, the clients who had the broken fix weren't putting the code into the same situation that my stress tests caused. The code change that caused the issue was in response to this issue relating to multiple threads calling WSASend() or WSARecv() at the same time. The solution was to add some per connection locking around the issuing of WSASend() and WSARecv() calls. Since there were several places in the code where this change was required it became a bit of a 'cut and paste' fix. The fix being applied to the TCP and UDP code in a similar pattern but in multiple places. One example of the change is this: const int result = ::WSARecv( socket.GetSocket(), buffer.GetWSABUF(), buffer.GetWSABUFCount(), &dwNumBytes, &dwFlags, NULL, // THIS makes us a sync read NULL); // In case the recv call fails, we have to obtain the last error before we unlock // the lock as otherwise the action of unlocking the lock will reset the last error // value... const DWORD lastError = ::WSAGetLastError(); #if (JETBYTE_TRACE_STREAM_SOCKET_READ_TO_DEBUG_LOG == 1) JetByteTools::W[...]



Hotfix now available for Slim Reader/Writer lock issue - Rambling Comments

2015-06-03T07:49:56Z

Back in September I mentioned that I had found a problem with my usage of Slim reader/writer locks. I expected this to be something that I was doing wrong but it turned out that it was a kernel bug. This...
Back in September I mentioned that I had found a problem with my usage of Slim reader/writer locks. I expected this to be something that I was doing wrong but it turned out that it was a kernel bug.

This morning Johan Torp tweeted that a hotfix for this issue is now available.

The note on the hotfix states: "Note These issues are not obvious and are difficult to debug. This update is intended for applicable systems to apply proactively in order to increase the reliability of these systems.", so go install it even if you don't know that you're having these problems right now...
(image)



Video streaming from IoT devices - Rambling Comments

2015-04-30T08:49:49Z

As I mentioned back in February I've been working on a custom video streaming solution for one of my clients. This has been a lot of fun. It was a nicely defined spec using industry standard protocols developed on a...

As I mentioned back in February I've been working on a custom video streaming solution for one of my clients. This has been a lot of fun. It was a nicely defined spec using industry standard protocols developed on a fixed price basis with lots of TDD, what's not to like.

The system allows a controlling application to set up RTSP streams for broadcasting to multiple clients. The data for the stream is being transmitted live from one of thousands of smart (IoT) devices and the server buffers this and broadcasts it using RTSP to publish the RTP streams. The controller can also get the server to record these live streams to disk and play them back via an RTSP stream at a later date.

All of the streams are dynamic and the controlling app is responsible for making sure that the IoT devices know where to send their raw video feeds and also for communicating the RTSP urls that can be used to access the streams.

Although this system was written using our new Streaming Media Option Pack there was a lot of scope for new code to be written. Up until this point the option pack revolved around playing pre-recorded data from files. Broadcasting and recording live video required a bit more thought.

This system allows for thousands of concurrent streams and allows multiple users to view what otherwise would be a single raw H264 stream from the IoT device. The device can stay reasonably simple and the server system can add the complexity required for broadcasting and conforming with the required specs.

To enable development without needing access to the custom hardware that's under development I put together a simple app which chops up a video file and sends it in the format that the IoT devices will use, this massively simplified testing. Given the amount of TDD that was being done on the actual code the integration testing with the mock IoT device was pretty painless but it made it possible to test the server under load and also allowed for a simple demo system to be put together.

I'd forgotten quite how much I enjoy these 'perfect' development projects where the customer knows what they want and can communicate it clearly, the schedule is sensible, the challenge is taxing but not impossible and the price is right. Throw in the comfort of being able to 'waste' as much time as I like with TDD and the end result is a project which actually ends up being quite a calming and meditative experience.

And now back to the real world...

(image)



New IoT fixed-price development project - Company News

2015-02-18T10:55:55Z

We're pleased to be working with one of our secretive security company clients again to provide a custom video streaming server for them. The server will enable them to record and stream live video from their network of "internet...
We're pleased to be working with one of our secretive security company clients again to provide a custom video streaming server for them. The server will enable them to record and stream live video from their network of "internet of things" connected devices.

As with all M2M and IoT projects this system needs to be able to scale to many thousands of concurrently connected devices and we're pleased to be able to use The Server Framework's new Streaming Media Option pack to be able to provide a complete solution quickly and cost effectively for our client.
(image)



That Slim Reader/Writer lock issue is a kernel bug - Rambling Comments

2015-06-03T07:35:33Z

It looks like the Slim Reader/Writer issue that I wrote about back in September is a kernel bug after all. Stefan Boberg has just tweeted to let me know that Microsoft has confirmed it's a bug and that a hot...
It looks like the Slim Reader/Writer issue that I wrote about back in September is a kernel bug after all.

Stefan Boberg has just tweeted to let me know that Microsoft has confirmed it's a bug and that a hot fix is in testing.
(image)



WSARecv, WSASend and thread safety - AsynchronousEvents

2015-01-19T09:57:33Z

Last week I learnt something new, which is always good. Unfortunately it was that for over 15 years I'd been working under a misconception about how an API worked. Tl;dr When using WSARecv() and WSASend() from multiple threads on... Last week I learnt something new, which is always good. Unfortunately it was that for over 15 years I'd been working under a misconception about how an API worked. Tl;dr When using WSARecv() and WSASend() from multiple threads on a single socket you must ensure that only one thread calls into the API at a given time. Failure to do so can cause buffer corruption. It all began with this question on StackOverflow and I dived in and gave my usual response. Unfortunately my usual response was wrong and after spending some time talking to the person who had asked the question and running their test code I realised that I've been mistaken about this for a long time. My view has always been that if you issue multiple WSARecv() calls on a single socket then you need to be aware that the completions may be handled out of sequence by the threads that you have servicing your I/O completion port. This is purely due to thread scheduling and the actual calls to WSARecv() are thread safe in themselves. I wrote about this here back in 2002. My belief was that you could safely call WSARecv() from multiple threads on the same socket at the same time and the only problem would be resequencing the reads once you processed them. Unfortunately this is incorrect as the example code attached to the question shows. The example code is somewhat contrived for a TCP socket in that it doesn't care about the sequencing of the read completions and it doesn't care about processing the TCP stream out of sequence. It issues multiple WSARecv() calls from multiple threads and the data being sent is simply a series of bytes where the next byte is the value of the current byte plus one and where we wrap back to zero after a 'max value' is reached. Such a stream with a max value of 7 would look like this: 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x00, 0x01, 0x02, 0x03... Validating such a TCP stream is as simple as taking any read that completes in any order and simply ensuring that the bytes that are contained in the buffer that has been returned follow the expected pattern. Starting from the first byte, whatever value it is, subsequent bytes must be one greater until the 'max value' is reached at which point the wrap to zero and continue to increment. Assuming my long held beliefs were true then it didn't matter how many threads were issuing WSARecv() calls on the socket at the same time, the resulting slices of the TCP stream should all be valid. Unfortunately the test program fails to prove this and instead proves that without synchronisation around the WSARecv() call the returned stream slices can be corrupt. That is the data in the buffers returned from the read completion can include bytes that do not follow the expected pattern. Of course the way that the test program uses the API is of limited use as I can't think of a TCP stream where it would be useful to be able to process the stream in randomly sized chunks with no need to have processed the data that comes before the current data. One of the reasons that I believed that I understood the requirements of the API was that I never used it this way. In systems where multiple reads on TCP streams were allowed I would alway[...]



Living with Gimpel Lint is made so much easier with Visual Lint - Rambling Comments

2015-01-07T16:12:38Z

I've been a big fan of Gimpel Lint for years. It's a great static analysis tool for C++ and it can locate all kinds of issues or potential issues in the code. My problem with it has always been that...
I've been a big fan of Gimpel Lint for years. It's a great static analysis tool for C++ and it can locate all kinds of issues or potential issues in the code. My problem with it has always been that it's a bit of a pig to configure and run, more so if you're used to working inside an IDE all the time. Several years back I had some custom Visual Studio menu items that I'd crufted up that ran Gimpel Lint on a file or a project and some more cruft that converted the output to something clickable in the IDE's output pane. This worked reasonably well but was complicated to maintain and easy to forget about. There was nothing to encourage you to run Gimpel Lint regularly and if you don't do that then the code starts to decay.

Visual Lint, from RiverBlade is, at heart, simply a Visual Studio plugin that integrates other static analysis tools into the Visual Studio IDE. However, the simplicity of the idea belies the value that it provides. Visual Lint makes Gimpel Lint a usable tool within the Visual Studio IDE. For me it has taken a very complex and hardly ever used tool which I always meant to use but never did and turned it into something that I use regularly and that is easy to run.

Gimpel Lint is still complicated to configure and it can be depressing when you first run it on a large codebase but once integrated with Visual Lint and run regularly it becomes a usable and powerful addition to your toolbox.

Now that I have Visual Lint running in my IDE and most of my code abiding by my in-house Gimpel Lint rules I suppose I should look at RiverBlade's LintProject tool and get my build servers to report on Lint breaks...
(image)



Two quite different approaches to multi-threading - Rambling Comments

2014-12-19T15:12:34Z

I've been working on some code for a client recently that needs to run in a multi-threaded environment. Unfortunately it was never really written with that requirement in mind, or, more likely, the people who wrote the code knew that... I've been working on some code for a client recently that needs to run in a multi-threaded environment. Unfortunately it was never really written with that requirement in mind, or, more likely, the people who wrote the code knew that it needed to be accessed from multiple threads but didn't really understand quite what that meant. As such I'm doing some fairly hairy refactoring for them. It's high risk as there are no unit tests and the budget doesn't really extend to completing the work, let alone "spending extra time" writing unit tests... The code is written in quite a 'C' style, most things are simple structs and most data is public and as such member functions fall where they are most conveniently written. Adding correct locking is complicated due to lots of mutable global data, lack of encapsulation and the fact that, at its heart, it's multi-user server software where one user gets to poke at all the others. Step zero was to realise that no locking is bad but correct locking was a long way away so I slipped in a single "global lock" to make the system work. It's stable and not too slow but it doesn't really ever use more than one thread at a time as almost everything requires that the executing thread holds the "global lock". Step one is to add some encapsulation and move things around so that we can one day dream of adding in more fine grained locking - unfortunately this step is a big one. I'm finding it interesting comparing this work to the work I was doing immediately before. I've been working on the next big release of The Server Framework and this release will put an Activatable Object at the heart of each connection. This removes the need for locking from the socket object and lots of stuff that's built on top of it. It means that masses of complex code can be pulled out and thrown away. It's not a new idea, the socket object is now simply driven by a message queue. Locks are needed to put work items into the queue but during processing we can run without locking, safe in the knowledge that only one thread can ever be processing at a time. The problem is that making the kind of change that I'm making right now is a massive change. It has led to a complete rewrite of the entire TCP side of The Server Framework. The general pattern is removing locking, because it's no longer needed and simplifying code because the ways in which it can be called are simpler. The unit tests that I had for the previous version of the code have helped to guide me; yes new tests have been written, but the old ones told me what I needed to test even if they needed to be changed a lot to work with the new code. With The Server Framework work it was scary but the various layers of tests supported the work and, in general, the code was being made simpler and easier to understand. With the client work there are no tests, huge amounts of the application simply didn't compile until the changes were complete and, whilst some things are being simplified, others are being made more complex to accommodate changes to make the code thread safe. I expect that with enough iterations the client's code could shift to using Activatable objects and some of the locking would go away agai[...]



How to build a GCC Cross-Compiler - Rambling Comments

2014-11-19T13:28:46Z

This article over on Preshing on Programming looks useful. It gives a step by step guide for building GCC cross-compilers, I expect it will save me lots of time at some point in the future.... This article over on Preshing on Programming looks useful. It gives a step by step guide for building GCC cross-compilers, I expect it will save me lots of time at some point in the future. (image)



Dropping support for Visual Studio 2005 and 2008 - AsynchronousEvents

2014-10-20T09:07:28Z

The 7.0 release of The Server Framework, which is likely to be released early next year, will no longer support Visual Studio 2005 or Visual Studio 2008. The 6.6.x releases will be the last to support these compilers. Please...
The 7.0 release of The Server Framework, which is likely to be released early next year, will no longer support Visual Studio 2005 or Visual Studio 2008.

The 6.6.x releases will be the last to support these compilers.

Please get in touch immediately if this will be a problem for you.
(image)



Dropping support for Windows XP and Server 2003 - AsynchronousEvents

2014-11-11T11:19:45Z

The 7.0 release of The Server Framework, which is likely to be released early next year, will no longer support Windows XP or Windows Server 2003. The 6.6.x releases will be the last to support these operating systems. Release...
The 7.0 release of The Server Framework, which is likely to be released early next year, will no longer support Windows XP or Windows Server 2003.

The 6.6.x releases will be the last to support these operating systems. Release 6.6.3, is due shortly and is a minor bug fixing release. We may release subsequent 6.6.x bug fix releases but no new development will occur on the 6.6 branch.

Removal of support for these operating systems allows us to clean up the code considerably and to remove lots of code that's required purely to work around 'interesting' twists in various Windows APIs pre-Vista.
(image)



New option pack: Streaming Media - AsynchronousEvents

2014-10-13T14:00:40Z

We have a new Option Pack, The Streaming Media Option Pack. This allows you to easily add streaming of H.264 and MPEG audio and video to your clients and servers using RTSP, RTP and RTCP. With more and more Internet...
We have a new Option Pack, The Streaming Media Option Pack. This allows you to easily add streaming of H.264 and MPEG audio and video to your clients and servers using RTSP, RTP and RTCP.

With more and more Internet Of Things devices supporting rich media streaming for remote monitoring it's becoming essential to have the ability to manage these media streams within your device management servers and clients. Whether it's recording device streams for later analysis or arbitrating between multiple clients and devices, manipulating streaming media is becoming more and more important.

As always, this Option Pack integrates seamlessly with The Server Framework's Core Framework and other options and allows you to quickly and easily add rich media support.
(image)



Surprising Slim Reader/Writer Lock thread exit issues. - Rambling Comments

2014-10-24T13:14:32Z

I've been noticing a strange thing for a while on Windows 8/8.1 and the equivalent server versions. The issue occurs when I'm using a Slim Reader/Writer Lock (SRWL) exclusively in exclusive mode (as a replacement for critical sections). What happens...

I've been noticing a strange thing for a while on Windows 8/8.1 and the equivalent server versions. The issue occurs when I'm using a Slim Reader/Writer Lock (SRWL) exclusively in exclusive mode (as a replacement for critical sections). What happens is, when a thread that has just unlocked a SRWL exits cleanly, immediately after unlocking the lock, sometimes threads that are waiting on the lock do not get woken and none of them acquire the lock.

At first I spent ages thinking that this was some kind of subtle bug in my LockExplorer tool as initially the problem only manifested itself during test runs that were using LockExplorer to detect potential deadlocks. Recently, however, I've been seeing the identical problem in normal programs being run with no clever lock instrumentation going on.

I still think it's more likely my bug than the operating system's but I recently found this Knowledge Base article, #2582203, "A process that is being terminated stops responding in Windows 7 or in Windows Server 2008 R2" which says "This issue occurs because the main thread that terminates the process tries to reactivate another thread that was terminated when the thread released an SRW lock. This race condition causes the process to stop responding.". This sounds suspiciously like the problem that I'm seeing, though not exactly.

The problem I see is that other threads waiting on the SWRL don't get notified that the lock is now unlocked. This locks my program up during shutdown purely because I have code that waits for these threads to exit cleanly and they can't as they're waiting on an unlocked SRWL. It's not a deadlocked attempt at recursive SWRL acquisition as I have debug code in my wrapper class which detects such behaviour. It's not an orphaned locked SRWL as breaking the hung process into the debugger and immediately continuing it causes one of the threads that's blocking on the lock to immediately acquire it and continue.

I haven't applied the hotfix yet, partly because my systems are Win 8 rather than Win 7 and partly because I'm not yet convinced that this is the issue.

As a work around I've added a call to Sleep at the point where my thread class allows a thread to exit. This seems to reduce the instances of the problem, which leads me to believe it's a race condition of some kind.

(image)



New client profile: Eonic Gaming - Turf Battles - AsynchronousEvents

2014-09-15T15:32:34Z

We have a new client profile available here for a new client who is using The Server Framework to power their MMORPG game server.... We have a new client profile available here for a new client who is using The Server Framework to power their MMORPG game server. (image)