A real breakpoint makes entries in the breakpoint filter for all offsets at the given line. Various flow control commands will also use the breakpoint filter to add and remove temporary "internal breakpoints" required during the flow operation. Ensure that we never remove a breakpoint filter entry if there was already one there due to a breakpoint (or really any other reason).
Flow control commands should not evaluate the conditions of conditional breakpoints when enabling and disabling breakpoints during stepping. Also, all breakpoints, rather than just the first matching breakpoint should be enabled/disabled.
There was a race in the debugger protocol tests which send bad commands to the proxy and expect it to disconnect. If the proxy thread was fast enough it would mark the client as stopped, and we'd skip trying to get a new message after sending the bad command. This means missing one line of output in very rare cases. I've put in a wait to ensure that the client settles in the stopped state before continuing, to remove the race.
break start/end/psp currently always report themselves as unbound. If the client is connected to a sanbox, these should instead be treated as bound. Also, break clear all currently removes breakpoints without running their destructors in the right order, which causes the break point counter to not reset to 1.
Fix a long-standing bug where if you step off of the end of pseudo main we'd segfault in the interpreter loop. To get this we have to have a breakpoint in the final TC in pseudo main, and have that called from another TC, then Step or Next off the end. We'd end up with a PC of zero after the ret, which makes sense, but the logic after interpreting a block of code to stay in the interpreter when debugging was blind to this case.
Added two simple functions to determine if a debugger is attached, and to provide info about where the debugger is connected from. Also fixed a minor bug in ext_socket where the name for an unbound AF_UNIX socket would come back from getsocketname()/getpeername() as garbage.
Added some test cases. Also now permit caller=>func() type of breakpoints. To make this work, I added a method, getCallingSite, to class InterruptSite and use this to walk the stack when checking if a site matches a break point that has more than one function name.
In general, a test which gets its input piped in is not really interacting with a terminal like it normally would. Thus, letting it think it has the same terminal as the environment is incorrect. More specifically, hphpd uses Readline, which inspects the TERM env var and based on that and a stunning array of other options will attempt to initialize the terminal with extra control characters on first use. It would be tough to try to control for all of the different options and ensure they're all the same, and honestly testing Readline is not the objective of our tests anyway. Thus, I'm forcing TERM to 'dumb' when a test has input to ensure a) a simple terminal which requires no control characters and b) consistency no matter what environment is present when the tests are run.
A recent diff called my attention to the logic in enterDebuggerDummyEnv()
and exitDebuggerDummyEnv() and I noticed it didn't look quite right.
The first time enterDebuggerDummyEnv() is called it creates a frame on an
empty call stack, but then exitDebuggerDummyEnv() does not correctly tear
down this frame and null out m_fp and m_pc, and this leads to subtle issues.
For example, invokeFunc() checks if m_fp is null to decide whether to call
enterVM() or reenterVM(). I found a case with the "flow_gen_excep.php" test
where invokeFunc was incorrectly calling reenterVM (because m_fp hadn't
been nulled out) and it was pushing bogus VM state info into m_nestedVMs.
This in turn was causing the logic in CmdNext::onBeginInterrupt() to get
confused when comparing the original stack depth with the current stack
depth.
This diff updates exitDebuggerDummyEnv() to correctly tear down the frame
and null out m_fp and m_pc, and it updates enterDebuggerDummyEnv() to
assume the callstack is always empty (which should be the case). I've also
beefed up the asserts in both of these methods.
After a run command, breakpoints in systemlib did not fire because systemlib.php does not get reloaded. Now get the run command to reapply the breakpoints. Also fixed the caching of the source of systemlib.php to not check for the debugger flag before the flag has been set. Finally, fixed a race condition where compiling a string/file to a unit will fail to update the repo with the unit because another thread has already written a unit with the same hash to the repo. In such cases, the losing unit is unable to retrieve source information because that only lives in the repo.
It turns out that Tutorial=0 means "auto" and -1 means "off". With auto tutorials, the hphpd.hdf file got updated and test behavior got flaky. Even worse, the non ASCII characters in the tutorial output broke the test runner in a way that made it look like tests passed when you put up a differential, only to fail in contbuild.
We've got some tests that are giving us control chars in their output on contbuild machines. We haven't been able to repo locally yet. Disabling them for now until the test harness can be improved to avoid crashing in these cases.
Note: this diff will likely go thru multiple iterations as more are uncovered.
These tests failed on the continuous build server, so we disabled them. Since then many other tests have failed and there is no good reason to suspect these tests of being especially flaky, so we may as well re-enable these tests.
It seems that on some machine configurations the readline library has the bad idea of spewing some control characters to stdout. This causes failures in any test that is reading from the command line, which currently only happens in debugger tests. Possibly this behavior shows up on the continuous build machines, so we'll make the expected output provide for this variation, just in case.
test/quick/debugger/break1.php and test/quick/debugger/break2.php are failing on the the continuous build machines. Disable these tests while we figure out what is going on.
Currently a breakpoint specified as relativefilepath:lineno will match only if the relative file path is a simple file name or if sandbox root + relative file path is an exact match. This is now generalized so that matching occurs if the relative file path is a suffix (path) of the absolute path of the execution location.
The debugger client now accepts feedback from the the debugger server about whether a breakpoint can be hit, absent further loads of files, classes or functions.
There was a similar-but-different event loop used when receiving command results from the server which was close, but not quite right. Unified it with the main event loop to ensure that all error cases are handled properly when we put up a prompt at a nested interrupt, like when hitting a breakpoint during an eval. The event loop is now shared, with a few different "kinds" to control some of the special needs of the loop when executed from a command. Most commands don't cause the server to run more PHP, so they don't change the machine state or cause more interrupts. But some do (Eval and Print) and certainly the top-level loop does, too. Made sure to throw a protocol error if any command causes this to happen when we don't expect it.
After a rebuild, the first run of the run/tests tests/quick/debugger/list.php invariably fails on developer machines. It passes if run again, or if another debugger test has passed before it has run.
Cleanup a lot of hangs with either the debugger client or server in a variety of error conditions, mostly related to communication errors or the client or server exiting unexpectedly. One of the biggest fixes is that all cases where the client was left in a state where Ctrl-C wouldn't work have been fixed.
Remove lots of little snippets of dead code. If you see a function (or small set of functions/fields) deleted then it was actually dead.
I debated whether to keep throwing DebuggerClientExitException on the server, and I decided to keep it. I think it's reasonable that if you've got the server stopped and you quit the debugger that the request gets terminated rather than continuing to run.
I also considered a big change to the way Ctrl-C works, but ended up staying with what was there with just a bit of cleanup. We need to guard against people banging on Ctrl-C, which is a reasonable behavior, and I think it feels pretty reasonable with the updated message.
Finally, added many comments about how this stuff works.
This reverts commit 2e9677b7c3f37e9627b9cbc9a6ddec82a10e7215.
Third time is the charm. I hid it from reflection, but I missed `get_class_methods`.
The diff betweenn this and what was reverted is https://phabricator.fb.com/P2217891 and then I did https://phabricator.fb.com/P2217904 because it looked like it should be done.
The debugger can only hit breakpoints and list source lines for code lines that have an entry in the byte code repository. In order to make systemlib.php debuggable, it needs an entry. A customized routine has been created for doing this since there is no actual source file available. Also, special case logic is needed to list the source file during debugging.
Small stepping, which is stepping over sub-expressions (kinda), worked but was a little goofy. The mode was set on the client, passed over with control flow commands, placed on the execution context, then retrieved from there and used by those same flow commands. Removed the execution context part of it, since it was useless, and factored grabbing the offsets into the flow cmds where they belong instead of doing it all the time.
The run cmd also had some notion of small stepping, but you'll note it was never sent over the wire. Nuked that, since it never mattered anyway.
Currently the debugger config file is always expected to be at ~/.hphpd.hdf, except when the debugger is controlled via the API, in which case the API client can specify the path. With these changes the path can now also be specified via the command line. Also, the test runner has been modified to look for an hphpd.hdf file and specify the path to it as a command line option. If present, this file also indicates that tests in the directory must be run in debugger mode, making the .opt files redundant. Note that it is important for tests to be able to control the contents of the config file since debugger output is controlled by it and tests will be brittle unless the config is part of the suite and not controlled by the user running the tests
There were multiple issues with flow control when exceptions occur. Fixed these by ditching the reliance on the exception thrown interrupt and introduce an exception handler interrupt, which indicates control is about to pass to a catch clause. This gives us much better insight into how execution is flowing and how we might need to adjust an in-flight stepping operation.
The debugger prompt string depends on user/machine specific configuration settings, which makes its inclusion in expected test output problematic. There is already an option in (the user controlled) ~/.hphpd.hdf to turn off the prompt. Now there is an option in normal config files to do the same.
Add reasonable behavior for stepping within continuations (generators). Stepping over a yield now does what one would expect. When the generator is driven from a C++ extension like ASIO, the next logical execution point is after the yield statement, and that's where we'll stop now. When driven from PHP, say in a loop calling send(), the next execution point is in fact the call site of send(), so we go there. Stepping off the end of the generator function, goes to the caller of send(), or the caller of the C++ extension. Stepping _out_ of a generator driven by a C++ extension ensures that we go to the caller and not back into the generator again. The logic for both cases is exactly the same. The difference comes from the fact that we don't actually debug C++ extensions.
This also fixes a long-standing problem where breakpoints would interfere with control flow cmds on the same source line. This caused funny behavior, like taking multiple steps to get off of a breakpoint.
We had a lot of odd behavior with both Next and Out. Previously the debugger would interpret the world until it saw that it was no longer on a specific line, or at a specific stack depth. I changed it recently to let the program run normally, and use an internal breakpoint to control step outs. Next's become like a step out temporarily when you descend into a function and need to get back out to the original line, so some bugs with out showed as bugs with Next, too.
Specifically, any time more PHP code was executed as a "side-effect" of a non-fcall instruction, step out would get lost. In these cases the stack trace gives us the offset of the instruction causing the PHP to run, not the instruction that control will return to as is the case with a Fcall. A breakpoint set there would get missed. This changes the step out logic to recognize such cases (via the fact that a nested VM state was pushed to execute the code) and step out more intelligently. We look at the instruction in question, and determine where execution may go, which might be multiple places. I also made a small change to ensure that we don't stop in generated functions when stepping out, which cleans up the iterator experience quite a bit, and sets us up for proper generator stepping, which will come next.
Expected test output for hphpd command line tests is difficult to read when colorization is present. Although there was an internal switch to turn off colorization (for the benefit of API clients), there was no command line option and no config file option to turn off colorization. This diff adds such an option and also fixes a few places where colorization was added regardless of the value of the internal switch.
Added a tweak to test/run to make it look for $test.in and, if there is one, to add <$test.in to the command line for running the test. Together with $file.opts and a config.hdf, this makes it possible to fire up a command line debugger for php, feed it with commands, obtain its output and check it against expected output.
Mike Magruder