2020. 1. 24. 12:55ㆍ카테고리 없음
Do you have calls to Thread.Sleep in your test code? If you’re not sure, you can easily find out by opening up the project in Visual Studio and running Find in Files (ctrl-shift F):Thread.Sleep will, not surprisingly, dramatically slow down your test suite. The primary reason is that it’s waiting longer than it has to, every time. Consider an example. You have an integration test that depends on an API call completing successfully.
In this article we will discuss how to put a c11 thread to sleep. C11 provides 2 functions for putting a thread to sleep i.e.
Usually the call takes a hundred milliseconds or so, but sometimes it takes a couple of seconds. You don’t want to have your test fail just because the call was a little slow, so you make sure it only fails when something is really wrong by adding. Sleep ( 5000 ); // sleep 5 secondsThis falls into the “better safe than sorry” category and at least lets you avoid having a test that intermittently fails for no apparent reason.
But it makes your test suite slow. If the call usually takes 100ms and you are waiting 5000ms every time, this test is now 50 times slower than it should be in the typical case. Instead of forcing your test to always wait a certain amount of time, you should write the test so that it waits up to that amount of time, but if the thing you’re waiting for happens, stop waiting! Using ManualResetEventThe ManualResetEvent lets you communicate between your asynchronous or multi-threaded code and your unit test. There are three simple steps to using ManualResetEvent in your test code.
Create the ManualResetEvent; pass false to its constructor – this indicates the signal you’re waiting for hasn’t yet been sent. Trigger the event when your async code completes by calling.Set. Wait to Assert in your test until the signal (Set is called) by calling WaitOne.
You can optionally supply a timeout.As a simple example, consider a case in which your test code needs to trigger an event that occurs on a separate thread, and then wait to ensure the callback occurs. One approach to this would be to trigger the event, then Thread.Sleep, but as I’ve already pointed out this is an evil antipattern. Instead, look at the following code, which tests the MemoryCache object in ASP.NET Core. The above test provides a lambda expression for the callback that has access to the ManualResetEvent, pause. Calling its Set method will immediately cause the call to WaitOne to return true. If the timeout value provided to WaitOne (500ms in this case) is reached, it will return false. In this way, the test never waits longer than it needs to for its dependent multi-threaded code to execute.
You can see several more examples like this one in the that I authored.Filed Under: Tagged With:, About Steve Smith.
I think it's important to note that this question is 8 years old, and application libraries have come quite a long way in the meantime. In the 'modern era' (2016) multi-threaded development comes up mainly in embedded systems. But if you're working on a desktop or phone app, explore the alternatives first. Application environments like.NET now include tools to manage or greatly simplify probably 90% of the common multi-threading scenarios.
C Thread Sleep In Linux
(asnync/await, PLinq, IObservable, the TPL.). Multi-threaded code is hard. If you don't reinvent the wheel, you don't have to retest it.–May 11 '16 at 15:06.
Look, there's no easy way to do this. I'm working on a project that is inherently multithreaded. Events come in from the operating system and I have to process them concurrently.The simplest way to deal with testing complex, multithreaded application code is this: If its too complex to test, you're doing it wrong. If you have a single instance that has multiple threads acting upon it, and you can't test situations where these threads step all over each other, then your design needs to be redone. Its both as simple and as complex as this.There are many ways to program for multithreading that avoids threads running through instances at the same time.
The simplest is to make all your objects immutable. Of course, that's not usually possible. So you have to identify those places in your design where threads interact with the same instance and reduce the number of those places. By doing this, you isolate a few classes where multithreading actually occurs, reducing the overall complexity of testing your system.But you have to realize that even by doing this you still can't test every situation where two threads step on each other. To do that, you'd have to run two threads concurrently in the same test, then control exactly what lines they are executing at any given moment. The best you can do is simulate this situation.
But this might require you to code specifically for testing, and that's at best a half step towards a true solution.Probably the best way to test code for threading issues is through static analysis of the code. If your threaded code doesn't follow a finite set of thread safe patterns, then you might have a problem. I believe Code Analysis in VS does contain some knowledge of threading, but probably not much.Look, as things stand currently (and probably will stand for a good time to come), the best way to test multithreaded apps is to reduce the complexity of threaded code as much as possible. Minimize areas where threads interact, test as best as possible, and use code analysis to identify danger areas.
It's been a while when this question was posted, but it's still not answered.' S answer is a good one.
I'll try going into more details.There is a way, which I practice for C# code. For unit tests you should be able to program reproducible tests, which is the biggest challenge in multithreaded code. So my answer aims toward forcing asynchronous code into a test harness, which works synchronously.It's an idea from Gerard Meszardos's book ' and is called 'Humble Object' (p. 695): You have to separate core logic code and anything which smells like asynchronous code from each other. This would result to a class for the core logic, which works synchronously.This puts you into the position to test the core logic code in a synchronous way.
You have absolute control over the timing of the calls you are doing on the core logic and thus can make reproducible tests. And this is your gain from separating core logic and asynchronous logic.This core logic needs be wrapped around by another class, which is responsible for receiving calls to the core logic asynchronously and delegates these calls to the core logic. Production code will only access the core logic via that class.
Because this class should only delegate calls, it's a very 'dumb' class without much logic. So you can keep your unit tests for this asychronous working class at a minimum.Anything above that (testing interaction between classes) are component tests. Also in this case, you should be able to have absolute control over timing, if you stick to the 'Humble Object' pattern. Tough one indeed! In my (C) unit tests, I've broken this down into several categories along the lines of the concurrency pattern used:.Unit tests for classes that operate in a single thread and aren't thread aware - easy, test as usual.Unit tests for (those that execute synchronized methods in the callers' thread of control) that expose a synchronized public API - instantiate multiple mock threads that exercise the API.
Construct scenarios that exercise internal conditions of the passive object. Include one longer running test that basically beats the heck out of it from multiple threads for a long period of time. This is unscientific I know but it does build confidence.Unit tests for (those that encapsulate their own thread or threads of control) - similar to #2 above with variations depending on the class design. Public API may be blocking or non-blocking, callers may obtain futures, data may arrive at queues or need to be dequeued. There are many combinations possible here; white box away. Still requires multiple mock threads to make calls to the object under test.As an aside:In internal developer training that I do, I teach the and these two patterns as the primary framework for thinking about and decomposing concurrency problems.
There's obviously more advanced concepts out there but I've found that this set of basics helps keep engineers out of the soup. It also leads to code that is more unit testable, as described above. I have faced this issue several times in recent years when writing thread handling code for several projects.
I'm providing a late answer because most of the other answers, while providing alternatives, do not actually answer the question about testing. My answer is addressed to the cases where there is no alternative to multithreaded code; I do cover code design issues for completeness, but also discuss unit testing.Writing testable multithreaded codeThe first thing to do is to separate your production thread handling code from all the code that does actual data processing.
That way, the data processing can be tested as singly threaded code, and the only thing the multithreaded code does is to coordinate threads.The second thing to remember is that bugs in multithreaded code are probabilistic; the bugs that manifest themselves least frequently are the bugs that will sneak through into production, will be difficult to reproduce even in production, and will thus cause the biggest problems. For this reason, the standard coding approach of writing the code quickly and then debugging it until it works is a bad idea for multithreaded code; it will result in code where the easy bugs are fixed and the dangerous bugs are still there.Instead, when writing multithreaded code, you must write the code with the attitude that you are going to avoid writing the bugs in the first place. If you have properly removed the data processing code, the thread handling code should be small enough - preferably a few lines, at worst a few dozen lines - that you have a chance of writing it without writing a bug, and certainly without writing many bugs, if you understand threading, take your time, and are careful.Writing unit tests for multithreaded codeOnce the multithreaded code is written as carefully as possible, it is still worthwhile writing tests for that code. The primary purpose of the tests is not so much to test for highly timing dependent race condition bugs - it's impossible to test for such race conditions repeatably - but rather to test that your locking strategy for preventing such bugs allows for multiple threads to interact as intended.To properly test correct locking behavior, a test must start multiple threads. To make the test repeatable, we want the interactions between the threads to happen in a predictable order. We don't want to externally synchronize the threads in the test, because that will mask bugs that could happen in production where the threads are not externally synchronized.
That leaves the use of timing delays for thread synchronization, which is the technique that I have used successfully whenever I've had to write tests of multithreaded code.If the delays are too short, then the test becomes fragile, because minor timing differences - say between different machines on which the tests may be run - may cause the timing to be off and the test to fail. What I've typically done is start with delays that cause test failures, increase the delays so that the test passes reliably on my development machine, and then double the delays beyond that so the test has a good chance of passing on other machines. This does mean that the test will take a macroscopic amount of time, though in my experience, careful test design can limit that time to no more than a dozen seconds. Since you shouldn't have very many places requiring thread coordination code in your application, that should be acceptable for your test suite.Finally, keep track of the number of bugs caught by your test.
If your test has 80% code coverage, it can be expected to catch about 80% of your bugs. If your test is well designed but finds no bugs, there's a reasonable chance that you don't have additional bugs that will only show up in production. If the test catches one or two bugs, you might still get lucky. Beyond that, and you may want to consider a careful review of or even a complete rewrite of your thread handling code, since it is likely that code still contains hidden bugs that will be very difficult to find until the code is in production, and very difficult to fix then. There are a few tools around that are quite good. Here is a summary of some of the Java ones.Some good static analysis tools include (gives some useful hints), (JPF & JPF2), and.is quite a good dynamic analysis tool (integrated into JUnit) where you have to set up your own test cases.from IBM Research is interesting. It instruments your code by inserting all kinds of thread modifying behaviours (e.g.
Sleep & yield) to try to uncover bugs randomly.is a really cool tool for modelling your Java (and other) components, but you need to have some useful framework. It is hard to use as is, but extremely powerful if you know how to use it.
Quite a few tools use SPIN underneath the hood.MultithreadedTC is probably the most mainstream, but some of the static analysis tools listed above are definitely worth looking at. Another way to (kinda) test threaded code, and very complex systems in general is through.It's not great, and it won't find everything, but its likely to be useful and its simple to do.Quote:Fuzz testing or fuzzing is a software testing technique that provides random data('fuzz') to the inputs of a program. If the program fails (for example, by crashing, or by failing built-in code assertions), the defects can be noted.
The great advantage of fuzz testing is that the test design is extremely simple, and free of preconceptions about system behavior.Fuzz testing is often used in large software development projects that employ black box testing. These projects usually have a budget to develop test tools, and fuzz testing is one of the techniques which offers a high benefit to cost ratio.However, fuzz testing is not a substitute for exhaustive testing or formal methods: it can only provide a random sample of the system's behavior, and in many cases passing a fuzz test may only demonstrate that a piece of software handles exceptions without crashing, rather than behaving correctly. Thus, fuzz testing can only be regarded as a bug-finding tool rather than an assurance of quality.