We had a lot of odd behavior with both Next and Out. Previously the debugger would interpret the world until it saw that it was no longer on a specific line, or at a specific stack depth. I changed it recently to let the program run normally, and use an internal breakpoint to control step outs. Next's become like a step out temporarily when you descend into a function and need to get back out to the original line, so some bugs with out showed as bugs with Next, too.
Specifically, any time more PHP code was executed as a "side-effect" of a non-fcall instruction, step out would get lost. In these cases the stack trace gives us the offset of the instruction causing the PHP to run, not the instruction that control will return to as is the case with a Fcall. A breakpoint set there would get missed. This changes the step out logic to recognize such cases (via the fact that a nested VM state was pushed to execute the code) and step out more intelligently. We look at the instruction in question, and determine where execution may go, which might be multiple places. I also made a small change to ensure that we don't stop in generated functions when stepping out, which cleans up the iterator experience quite a bit, and sets us up for proper generator stepping, which will come next.
runtime/eval is a relic of a bygone era. As long as we're cleaning up
our directory structure, let's move FileRepository (the only remaining
thing in runtime/eval/runtime) to where it makes sense.
runtime/eval still contains the debugger, which would probably make more
sense as runtime/debugger, but I don't want to throw a wrench in the
works for @mikemag and @hermanv unnecessarily.
Expected test output for hphpd command line tests is difficult to read when colorization is present. Although there was an internal switch to turn off colorization (for the benefit of API clients), there was no command line option and no config file option to turn off colorization. This diff adds such an option and also fixes a few places where colorization was added regardless of the value of the internal switch.
- boost::shared_ptr now has "explicit operator bool", which means we
can't "return <a shared ptr>" from a function with return type bool.
- Our use of shared_ptr<T[]> in debugger_client.h was screwed.
Unfortunately it's not straightforward to make it a unique_ptr as per
our discussion; there are other objects that call setLiveLists, and
it's not clear to me that this is a total transfer of ownership. I'm
just fixing the immediate problem using shared_array. (Also made the
typedef less confusing, hopefully.)
- Our specialization of graph_traits<G> for ControlFlowGraph didn't
define the null_vertex member function. This didn't matter in older
versions of boost because they didn't use it internally, but now they
do (the call to depth_first_search in control_flow.cpp was failing to
compile).
Functions like array_filter and array_map need "withRef" semantics,
and while it can be simulated in php via copy-on-write, doing so
is very inefficient.
This adds
SetWithRefLM
SetWithRefRM
- similar to SetM but binds the value if it was a reference. L reads a local, R reads a return value
WIterInit
WIterInitK
WIterNext
WIterNextK
- essentially the same as the corresponding opcodes without W, but the value local is set by reference if the array element was a reference.
The proximal purpose of this differential is to fix the debugger client so that it does not write raw strings to the console (with control characters doing their thing to the console and unprintable characters pretending not to exist). This change in behavior means that the serializer used to convert values to strings can no longer be instantiated with the PrintR option, as it has been in a subset of cases. (The reason being that print_r() must not change its behavior.)
Rather than introduce yet another serializer option, I decided to always make the debugger client serialize user visible values using the existing DebuggerDump option, which is already used in a number of such cases and which has no other use. To make the overall change less painful for users, I preferred to change behavior of DebuggerDump to be more like PrintR in a number of cases, primarily the formatting of objects (where the current behavior is to make a JSON like string).
This does not impact the behavior of the debugger client API or the behavior of FBIDE, since these do their own thing directly with the DebuggerSerialization format and never see the result of the DebuggerDump format.
In runtime code, use StaticString for literals or String for computed
char strings. In test code, I justed used String("literal") to keep
the conversion simpler. I think tests are ultimately easier to maintain
without the StaticString treatment.
The special case got over looked in the recent breakpoint parser rewrite. This change is a small point fix that restores this functionality while perturbing as little else as possible.
"hhvm -m debug" without specifying a file should work without raising
notices. hphp_invoke_simple() and friends used to (incorrectly) tolerate
using an empty string for the file name, and hphpd was relying on this.
This diff fixes hphpd to not call hphp_invoke_simple() when no file was
given at the command line.
Log details about debugger usage from both client and server side. Added a new runtime option to control whether the logging is used or not. There's a base class for a usage logger defined and used under runtime/eval/debugger.
I'm adding the count of connected debuggers, and whether or not the process is hphpd, to the crash reports. In another diff I'll wire these up to hphpcrash_categorizer.py and get these as columns in the Hphpcrash Scuba data set so we can filter crash reports based on whether or not it's from hphpd, and whether or not a server was being debugged when it crashed.
This is a rather mechanical refactor that uses references (&) rather than pointers (*) for parameters that are not permitted to ever be given null arguments. In effect, the onus for checking null pointers is shifted from the callee to the caller. The & type annotation makes it clear that the callee is not prepared to deal with a null pointer.
This adds a new trace macro to allow tracing to the ring buffer in release builds. It also performs normal tracing, too, like TRACE(n, ...). Converted a number of trace/log messages in the debugger to the new macro so we have this data when we get a core file. I've converted things that we've found useful lately, but this will be adjusted over time quite a lot as we discover new things that help us find problems more quickly, or find messages that turn our to be useless or spammy.
A call to hphpd_break() with no debugger attached, from a command-line run (no request), with debugging enabled (not normally true, but can be true if you use something besides cli.hdf), will segfault every time. Simple fix for a high hit-count crash.
These commands invariably return true. This is more than a tad confusing to a new reader of the code. Refactor them to return void. Remove a few pieces of dead code that would log something if they ever returned false.
The error handling around the communication channel between client and server is a bit sketchy right now. We have some cases that are reasonable when the client is quitting and the socket is closing, and some cases where it would be surprising for this to happen. This leads to spurious messages printed out when quitting the client normally. For now, switch some of these to trace messages instead of log messages. Task 2398988 tracks cleaning this up properly.
Breaking at the start of the request, end of the request, or end of PSP has been broken for a while. Giving these events a site with the proper URL in them ensures that the rest of the original logic to match and stop on these breakpoints works again. Also updated the help for the breakpoint cmd using the old text from the reference, which seemed reasonable.
I was learning from @jdelong and he said that you should use
double quotes for local includes and angle brackets for library
includes. I asked why our code was the way it was, and he said he wanted
to clean it up. I beat him to it :)
Conflicts:
hphp/runtime/base/server/admin_request_handler.cpp
hphp/runtime/vm/named_entity.h
Set the stopped flag for the debugger proxy when it receives a quit command, so that the proxy exits a bit quicker and cleaner. Add a timeout to the cleanup quite of test_debugger, so that it does not hang when a test goes wrong. Add some more tracing.
The debugger's Instrument command does not work with the JIT; any instrumentation requests will be ignored for jitted code. We believe that no one is using this at this time. I'm starting with a small diff to disable the command and remove the impact it has on the interpreter for now, just in case we're wrong. Once the command has been disabled for a few weeks I'll come back and remove all of the code (task 2376711).
InstHelpers was just pure dead code, so I nuked it.
Added comments to every method in cmd_breakpoint.cpp. Also renamed some methods to make their intent more obvious. Moved a few implementation methods from the public to protected space.
The specification string following a break command was parsed by means of ad-hoc string splitting and sub string finding. This scheme cannot deal with namespaces and class names that contain : (such as xhp classes). There is now a proper parser for break point specifications. Also, xhp class names are mangled before putting them in the BreakPointInfo structure, otherwise breakpoints qualified with xhp classes will not be hit. The syntax for namespaces has been changed to fit the language and if present namespaces are used to mangle function and class names, so that they match the names produced by the regular parser.
Added some more logging on the server when debugging. Log warnings during shutdown (normal and from the signal handler) when there are debuggers attached. Minor tweaks to some of the error handling paths, with more log messages instead of silent failures. Also moved server-side closing of the thrift buffer to proxy destruction.
Rather than test RuntimeOption::EvalJit and 5 thread locals to determine
whether or not to run the jit on each re-entry, maintain one thread local.
Make various RequestInjectionData fields private to ensure that the jit
flag is kept in sync.
This cleans up the code a lot, and takes it out of various hot
paths. It will impact perf for requests where intercepts are used,
but no longer penalizes requests that don't use intercepts.
This improves both Next and Out to avoid interpreting and stepping everything between when they start and finish. Out now lets the program run free until a pseudo-breakpoint at the return site it hit. Next continues to single-step the source line being stepped over, but now lets the program run free under calls made from that line.
The logic in Next regarding "calls made from that line" is extremely generic. We don't look at, say, call opcodes and decide to do something special. Rather, when we find we're off the original source line and a frame deeper we setup a "step out" operation much like the Out command then let the program run free. When we reach our return point, we continue stepping like normal. This accounts for not just calls, but iterators, and anything else that causes more PHP to run under the original source line.
This change moves the flow control logic down in to the respective cmds: Next, Step, Out, Continue. These cmds get a crack at executing at various points in the interrupt/command processing path. These cmds now own setting up the last location filter, whether they need VM interrupts, and whether they're done or not.
Update a number of things to make optionally generating the parser at
build time possible. @sgolemon will add the OSS pieces of this in a
separate commit.
Removed the litstr overloads of Array::rvalAt, rvalAtRef, lval,
lvalPtr, lvalAt, and set. The main one left to do is operator[].
Fixed a bug in f_get_html_translation_table() where we were copying
the null terminator of what should be a one-character string, thus
creating a two-character string with s[1] == 0. (cc @jdelong)
In class Extension, store a String for the name instead of
const char*. cc @sgolemon
This gets rid of the (litstr) StringData and StackStringData
constructors, but keeps String(litstr). Also rename all
the instances of AttachLiteral to CopyString, since they now
mean the same thing.
When debugging a script using a local VM (i.e. when there is no remote VM),
the cleanup actions performed by the run command caused the debugger client
to shut down as well. Since this seems to be intentional, processing of the
run command now restarts the debugger client and proxy. Also, following a
ctrl-C, the m_lastLocFilter field must be cleared, otherwise an empty endless
loop cannot be broken into for a second time.
The separation between these classes was a vestige of days gone by. Combined them. I ran into issue with having the proxy split up in particular when working on stepping, so doing this now as a separate diff to keep things cleaner.
Cleaned up some of the superfluous variations of onServer*()/onClient*() in the debugger commands. The separation was a leftover of the days when we had the VM and the compiler. The entire DebuggerCommand interface, what's public/protected/private, etc. could use a serious cleanup, but I'm not going that far now. I mostly wanted to clean this up to make some other work we need to do server-side less complicated.
This is a partial step towards merging the HPHP::VM namespace
up into its parent. To keep it reviewable/mergeable I'm not doing
everything at once here, but most of the code I've touched seems
improved. I've drawn an invisible line around the jit, Unit and
its cohort (Class, Func, PreClass, etc.); we'll get back to them
soon.
Hphpd's Jump command has been fundamentally broken for a long time. It was originally implemented to run the byte code in a modified way which didn't make state changes, and wait for the destination offset to be reached. We lost the ability to do that long ago, and the implementation of this command has atrophied since. As it stands now, if you're lucky it might act like "run until", which is the opposite of what it is documented to do.
I've removed the command entirely. Fixing it is a very large effort which we might consider some time in the future.
I found myself getting confused, thinking that proxy->send(cmd) sends a command to the proxy, when in fact it causes the proxy to send the command to the client. These are now named sendToClient, sendToServer and recvFromServer so that future readers (including my future self) will not make this mistake again.
Owen Yamauchi