We force users to use the create() methods of StaticResultWaitHandle and StaticExceptionWaitHandle, which properly set m_resultOrException. However, deserialization will use the normal constructor which was failing to initialize the field. The destructor, then, would operate on random data. This broke hphpd when a stack trace had one of these objects on it somewhere. The 'where' command would succeed, but the next command (which deletes the stack trace), would segfault in the wait handle's destructor.
I think it's fair to not serialize this member. It's not exposed thru PHP except via joining with the wait handle, and I think an argument can be made that it's just plain wrong to join with a deserialized copy of a wait handle. So I've just initialized it to a reasonable default.
Turns out I need to use tvSet with temporaries. Also removes
unused tvSetObject*, and replaces some tvSetNull* callsites with using
make_tv temporaries.
Since TypedValue::operator= is dangerous, something like
tvTeleport is usually what you want to use, but it doesn't work with
temporary TypedValues (e.g. return values of things like make_tv or
cellAdd), because it took the arguments by pointer. This also means
we can change it to take parameters by value later without updating
callsites. This diff does the tvDup family and changes tvTeleport to
tvCopy. I'll gradually get the other ones done, but I just need these
for now to work with temporaries for changing SetOp to not use Variant
arithmetic.
Updates continuations to allow yielding of a key-value
pair from a generator. Adds bytecode instructions (PackContK,
ContKey) for using the new feature, and adds IR instructions
(ContUpdateIdx, ContIncKey) to help get it down to the metal
(in particular, ContIncKey attempts to keep the current use-cases
as fast as possible).
More changes for HPHP to help make it clang friendly
~~~hphp/compiler/expression/constant_expression.h
~~~hphp/compiler/expression/function_call.h
rfind returns a size_t/unsigned int
~~~hphp/runtime/base/server/http_protocol.cpp
Switched to std::to_string. Assuming [] was not intended here
~~~hphp/runtime/base/ref_data.h
These fields were accessed in a public manner, assuming public was intended
instead of private
~~~hphp/runtime/base/variable_serializer.cpp
Switched to using [] and & to make clang happy. Assuming this was to either
take or drop the first char.
~~~hphp/runtime/ext/asio/asio_external_thread_event_queue.h
~~~hphp/runtime/ext/asio/asio_external_thread_event_queue.cpp
Cast which performs the conversions of a reinterpret_cast is not
allowed in a constant expression. This is been moved to a macro as a temporary
fix.
+++hphp/runtime/ext/ext_misc.cpp
Added std::atomic to supress warnings
~~~hphp/runtime/vm/jit/simplifier.cpp
Chosen constructor is explicit in copy-initialization
~~~hphp/runtime/vm/jit/translator-asm-helpers.S
Ambiguous instructions require an explicit suffix
Changed cmp to cmpl
~~~hphp/runtime/vm/jit/translator-x64-helpers.cpp
Clang does not support global register variables
+++hphp/runtime/vm/unwind.cpp
describeFault was only used when DEBUG or USE_TRACE was defined
~~~hphp/runtime/vm/verifier/check_unit.cpp
Made fmt pointer const to avoid string format issues/warnings
~~~hphp/util/stack_trace.cpp
Clang does not support variable-length arrays.
Uniqe_ptr is used instead to take advantage runtime-sized arrays, a
restriced form of variable-length arrays
~~~hphp/util/thread_local.h
Clang seems to be supporting the __thread attribute, or at the very least
it is not complaining about it.
~~~hphp/util/tiny_vector.h
Clang does not like the flexible array here, since T is not always POD.
I have reimplemented the array here by just sticking one value in the struct
and calculating the offset from its address manually.
Alterinatively, we could change the the non-POD types to be pointers, or
we could edit their implemenations.
+++hphp/util/util.h
Created a template for the union,
A function declared with the constexpr specifier cannot contain
type declarations that do not define classes or enumerations
+++hphp/runtime/vm/jit/x64-util.h
Added a TODO
The way hphp/runtime/vm/jit/x64-util.h is currently implemented, it only
works if USE_GCC_FAST_TLS is defined
AsioExternalThreadEvent::unserialize() may legally throw PHP exceptions.
When that happens, construction of PHP exceptions reenters VM, surprised
flag is checked and pending C++ exceptions may be thrown.
Let's make sure the queue of received events in runUntil() is not lost.
Move this responsibility to the AsioExternalThreadEventQueue.
External thread event queueing logic is big enough that it deserves its
own class. Move the code and rename members to better reflect the
underlying operations. Implementation is otherwise unchanged.
AsioExternalThreadEvent::unserialize() may legally throw PHP exceptions.
When that happens, construction of PHP exceptions reenters VM, surprised
flag is checked and pending C++ exceptions may be thrown.
Let's make sure the event is properly destroyed. If it is not, the web
request will get stuck forever waiting for the finished event to finish.
I noticed that directorty structure of hphp/system was a bit scattered, so
I consolidated things to reduce the total number of folders and to put
related things together with each other.
This diff moves the contents of "hphp/system/classes_hhvm" into
"hphp/system", it moves the contents of "hphp/system/lib" into
"hphp/system", moves "hphp/idl" to "hphp/system/idl", and moves the
contents of "hphp/system/globals" into "hphp/system/idl".
Recover from C++ exceptions so that PSP does not think ASIO is running. Fixes a segfault when ASIO fails on internal invariant violation in runUntil() with m_current == nullptr.
- smart::make_unique, returns a unique pointer to smart allocated
data.
- Support forwarding in our allocator's construct, so
smart::vector<foo::unique_ptr<>> works.
- Move smart containers to their own header.
A race condition can occur when the processing thread tries to notify
condition that is not yet waited upon. This can happen during the short
window in the web request thread between transitioning to the WAITING
state and calling condition.wait().
Fix this by locking mutex right before notifying the condition. If the
web request thread is in WAITING state, it is guaranteed to hold the
mutex or wait for condition variable. If the processing thread was able
to transition away from the WAITING state and grabbed the lock, the web
request thread must be waiting on condition variable, so it is safe to
notify it.
Thanks @andrii for tracking the bug down to ExternalThreadEvents and
providing a workaround (grabbing the mutex few lines up than this diff).
Destructor of AsioExternalThreadEvent uses assert() to check whether the
state is legal. Introduction of abandon() mechanism makes it possible to
destroy the AsioExternalThreadEvent in the Waiting state and hit the
assertion error.
This diff solves the problem by introducing Abandoned state.
I was learning from @jdelong and he said that you should use
double quotes for local includes and angle brackets for library
includes. I asked why our code was the way it was, and he said he wanted
to clean it up. I beat him to it :)
Conflicts:
hphp/runtime/base/server/admin_request_handler.cpp
hphp/runtime/vm/named_entity.h
Sometimes, a PHP object (usually defined by extension) is needed to
unserialize the data.
Provide a safe mechanism to store such object with external thread
events.
In order to build a dependency graph of continuation execution and data-fetching in PHP-land, we need a few instrumentation points in the asio_ext HHVM extension. There are 4 additions required:
1. Callback when a continuation finishes successfully.
2. Callback when a continuation blocks on a wait_handle.
3. Get array of WaitHandles a GenArrayWaitHandle is waiting on.
4. Get WaitHandle that the SetResultToRefWaitHandle is waiting on.
I don't think this should really affect performance, as in the normal case, nothing has changed, but you never know... I'm also not sure who should be reviewing this, so I've just added @jan for now. If you could pile other people on, that would be cool.
sandcastle appears to be broken.
For cppext static methods we have a "ti_" method that takes an extra unused
parameter "const char* cls" and then a "t_" wrapper method that calls the
"ti_" method.
This makes things unnecessarily complicated when writing HipHop extensions.
Let's get rid of the extra unused parameter "cls" and the unnecessary "t_"
wrapper method.
ContinuationWaitHandle::start() was deprecated in favor of Awaitable
interface's getWaitHandle().
Remove ContinuationWaitHandle::start() from public API and rename it
internally to c_ContinuationWaitHandle::Create() that is called only from
Continuation's getWaitHandle(). Since it is guaranteed that the
Continuation does not have associated any wait handle, we can remove the
code that handles that case.
Add support for external thread events.
See asio_external_thread_event.h for details about usage.
This is the first version that compiled, the code was not executed even
once, but wanted to get it out early to unblock @hannesr. Thus the [RFC]
flag.
My next step is to implement asio-compatible equivalent of
fb_call_user_func_async() that will make it possible to easily test the
new functonality by simulating async stuff with sleep(), etc. on the PHP
side.
There are 2 major mostly lockless synchronization points that needs
careful review:
- queue of ready external thread events (asio_session.{cpp.h})
- shutdown cleanup (asio_external_thread_event.{cpp,h})
This is a near-straight translation of gen_ext_hhvm.php into C++. The
main goal of this change is to break the bootstrap cycle in the build
process: we have to run PHP scripts to build our PHP interpreter. This
isn't so bad for us internally, but in the open-source build, it's
philosophically lame to depend on an external PHP interpreter (i.e.
Zend) to build HHVM. To get around this, we have to check in the
generated ext_hhvm files: no good.
There's nothing particularly sophisticated in this program; it's just a
bunch of grunt work. I took the opportunity to make a few improvements
to the generated code -- there was some int-width confusion, and a bunch of
duplicated code to transfer return values to the right place.
This is a partial step towards merging the HPHP::VM namespace
up into its parent. To keep it reviewable/mergeable I'm not doing
everything at once here, but most of the code I've touched seems
improved. I've drawn an invisible line around the jit, Unit and
its cohort (Class, Func, PreClass, etc.); we'll get back to them
soon.
I was trying to simplify code flow by removing things with tainted variables. I got too carried away and thought tainted wait handles were the same.
goodcatch
@override-unit-failures
All m_* object members in ext_asio are camelCase, except in AsioContext.
Rename them. Also, m_queue_ready became m_runnableQueue, as with the
introduction of async functions a new abstract RunnableWaitHandle will
be introduced.
While I was working on the TestCodeRun refactor I found two tests about Tainted code. I looked into it and coulnd't get HHVM to compile with TAINTED=1. Then I checked and none of the extension functions we exposed about tainting were used in WWW. Scratching my head I asked, @srenfro and @jdelong, who thought it was dead. So I killed this zombie.
unserialize() and call_user_func_array() were straightforward. They were
called from all over the runtime, but I renamed those implementations
and codemodded the runtime.
The is_* functions were only ever being called by the CVarRef signature,
so I deleted all the other ones (same for f_gettype). Only some of the
is_* functions were being called from the runtime, so I made inline
versions of those without the f_ prefix.
Provide a Static{Result,Exception}WaitHandle::Create() static method for
internal use, use it by GenArrayWaitHandle.
Avoids one inc/decref (Array -> Variant -> m_resultOrException) if
GenArray received an array of finished dependencies.
If a GenArrayWaitHandle is imported, the current implementation imports
only the active child. Try to import other children as well to increase
parallelism.
enterContext() can throw an exception if a cross-context dependency
cycle is found. It is safe to ignore such exception when importing
non-active children. The import will be attempted and fail again once
onBlocked() reaches the dependency that causes the cycle.
Verify sanity of input array of dependencies in advance. Callers get the
error earlier and it also makes it easier to work with the array outside
of the main loop.
Our ext_hhvm generated code is casting TypedValue* to Value*
on the assumption that the offset of TypedValue::m_data is 0.
Fix this assumption, and also while in the same code, replace
some (t == KindOfString || t == KindOfStaticString) with
IS_STATIC_STRING(t), which does a single bit test instead of
two comparisons.
In an unlikely situation a user of ext_asio may tamper with Continuation
before passing it to the ext_asio extension. Let's fail with an
exception if this happens. Previously, a user bug would stay unnoticed,
but would not harm the ext_asio code.
This check will be needed once we implement optimistic execution. With
optimistic execution, we iterate continuation and defer construction of
ContinuationWaitHandle until the first blocking event occurs. During
this phase, a standard dependency loop detection is skipped and the code
would try to iterate a continuation that is already being iterated.
Edwin Smith