| |
|
|
|
| |
|
| |
|
|
| |
What's News in the PlayBasic world ? - Find out here.
|
PlayBasic To DLL Update
|
|
By: Kevin Picone Added: April 2nd, 2014
|
|
Category: All,Machine Code,Tools
|
The following is from a selection of blog posts from the development of PlayBasic to DLL (Machine code translator) over the past months.
PlayBasic To Dll - User Define Types - Addressing the same ground. (December 15, 2013)
I've been testing the translation tonight hunting down those little gotchas in some bare bones type demos. Found one such issue where PB would crash on exit when a typed array created inside the dll was returned to the VM. Initially was thinking it was caused due to some difference between the Undim process in the VM, which is called when the VM exists. This was a close guess, but it turned out to be just the order in which some of the vm libraries release themselves. VM would release any bound resources such as DLL's before it release the array memory. This is no problem when all the array data is created locally in the VM, but it's a big issue when the arrays point to structure definitions inside the DLL. Which would cause access violations and hence the crash.
Today's build gets us to a point where the DLL side and VM side can create a type and other side can release it. This allows the flow of data from VM to DLL and DLL to VM more freely. Even so, i'd still recommend at least trying to keep the data life cycle management to one side or the other, but it's workable if you can't wrap your head around that. If this all sounds familiar it's because a few months back I had to rewrite how the VM deals with it's local types. In the older editions the VM maintains one central list. Which is fine in a world where 3rd party code isn't expected to plug into the VM, but bound DLL's need to be able to control data much the same way the vm does. There's other solutions of course, but altering it to support host structures seems the most workable.
All this messing around means you can finally do this, bellow we have a function that we'd export to Dll and bellow that is some code that accepts the returned structure..
// Declare structure on DLL side..
// The VM will need a copy of this also if you
// indent to look at the fields in the types..
Type Dude2
xxxx,yyyy,zzzz
string$
endtype
Function DLL_FillArray(Size)
Dim Cool(size) as Dude2
For lp =0 to size
Cool(lp)= new dude2
Cool(lp).xxxx=100+lp
Cool(lp).yyyy=200+lp
Cool(lp).zzzz=300+lp
Cool(lp).string$="This String "+Str$(lp)
next
EndFunction Cool() as Dude2
The dll code just creates a junk array and fills it with some data, before returning the array handle upon completion back to the caller, the VM side.
linkDLL "testdll.dll"
DLL_FillArray(Size) alias "fillarray" as integer
EndLinkDLL
// Declare structure on VM side
Type Dude2
x,y,z
Name$
endtype
// Dim our reciever array as type DUDe2
Dim K(0) as dude2
// Call our DLL function and assign the array handle to our K() array
k()= DLL_FillArray(10)
// query the contents of it
if GetArrayStatus(k())
for lp=0 to GetArrayElements(k())
print k(lp)
print k(lp).x
print k(lp).y
print k(lp).z
print k(lp).name$
next
endif
print "done"
Sync
waitnokey
waitkey
The VM side is just running through the contents of the array and dumping them to the screen. Not much of an example, but it's fairly seamless. There's a couple of oddity that appear in the debugger if you looked at the contents of the k() array in the VM after it was created externally, but those seem from the DLL's dimension instruction not classifying the arrays structure. So it doesn't know the original 'name' of the type.
Edit #1 - Debugger Name Resolving
One problem i'd been having with structures passed back from a DLL was that if you clicked on them in the debugger the structure names would be wrong, but that seems to be now fixed.. As if you got to variables and click on k() array, it drops the info into the panel as normal.
PlayBasic To Dll - Redim - Life cycle of arrays & LIsts ( December 18, 2013)
Pretty much all PB's data structures uses the same interface, that being the array interface. Most of the legacy code i'm ok with, some of it i'm updating to bring in some of the newer ideas explored in it's descendants. Haven't had much time to really chomp through it, but what I have been slowly working up the core routine with their support code. One such replacement routine was the ReDIM function, which is a lot more complicated than the DIM operation.
Dim is fairly dumb, it just creates some empty (formatted) space for the array. There's a little more to it than that, but conceptually that's about it. ReDIM on the other had has to rebuild the structure when the sizes are different. If you make an array larger, then it allocates some new memory and copies the old data across, the same goes for shrinking an array. It copies the existing data,not forgetting to deletes any excess. Which can be an easy way to leak resources if you're not careful. See-> Tips On Safe Resource Management:
The other thing i've been working on is the back end support for the linked lists. A link list is an optional extension to the array structure, so when a list is present the address resolver uses the structures internal current position. So far it's in a very bare bones state, due to how interdependent some of the back end actually is when dealing with lists. For example the list iterators are set up, but insertions don't work because that requires insertion management of the arrays size, hence the need for ReDIM.
But here's a general idea of where it was at last night..
Function DLL_ResizeArray(Size)
Dim Cool(Size)
For lp =0 to Size
Cool(lp)=1+Rnd(10000)
next
redim Cool(Size+10)
EndFUnction COol()
Type Dude2
xxxx,yyyy,zzzz
string$
endtype
Function Dll_TestLIst(Size)
Dim Z as dude2
Dim L as dude2 list
For lp=1 to size
l=new dude2
l.xxxx=lp
print l.xxxx
next
EndFunction L.Dude2
I think it should be smoother sailing once all the little inter-dependencies have been ironed out.
PlayBasic To Dll - Macros make the world turn faster
The back end assembly is built with flat assembler which has some very impressive macro support, most of which is way beyond the scope of the project and my understanding of it  . However, macro's can be a very powerful approach to programming, in particular in assembly, as they allows the user to dramatically reduce the lines of code, where a common behavior is wrapped as a macro and used throughout the project. It's a bit like a function, except the macros are parsed out by the assembler at compile time. So all the code is inlined.
So what i've been doing is moving more of the code in the base template into macros. You could pretty much make macro's for all the core operations, which would simplify PlayBasic to Dll's job by hiding how something works away in the template. The current build of the generator is pretty reactionary, so when it sees some known byte code pattern it spits out the same' template assembly chunk in reply. Such things could well be macro'd, allowing improvement to be make to the template and not only via the translator.
PlayBasic To Dll - Data Statements (January 22, 2014)
There's only a few tidbits left in the entire conversion felt to do and DATA statement support is one of them. Initially I wasn't going to bother with them, in favor of resource binding, but local data is probably easier for most people to wrap their heads around. There's a few gotcha's with exporting data to our DLL's though and that's visibility. In the VM data is global regardless of where it's defined, so the data heap exists outside of the program code. Now if you compile a PlayBasic source to DLL that has Data in it, now the question is who should have access to that data.
It's viable that the VM could retrieve any local data from the a bound dll and pull it into the global pool on the VM side, but then the dll's have to use the VM access. Which is about as painful as it sounds ! - I think i'd much rather that any data be local to it's dll, which makes a lot more sense in terms of encapsulation anyway. If the dll's shared data with the main program this could affect the users code in the VM or even in other bound dll's. Making it a slippery slope.
The DATA command set was re-written for the VM3 prototypes, but after looking over the replacement code, there's a few areas where it could be further improved. I think some of those changes would have to be made at compiler level to really see the benefits, but one thing that comes to mind would be replacing how the string heap is set up. If you have a lot of string data in a program and those strings share fragments/words, then there's some lost opportunities in terms of compression.
Ie.
Data "Hello World", "Hello World2"
The above data fragment would store two strings, since they're not a perfect match. But what it could do at build time is compare smaller fragments with the raw pool. So any matching runs would compress away, both still exist, it's just one is a part of the other.
Stuff like this doesn't make any significant difference to small programs, but as programs get bigger the amount of partial collisions increases.
PlayBasic To Dll - Function Calling Cont. (February 03, 2014)
Yesterday was get function calling done day, but things didn't turn out how I'd hoped. In fact spent almost 12 hours on it, only to run into brick wall after brick wall. Was initially going to wrap up some functionality in the library to handle an emulated VM stack, which I know would work and would be a perfectly fine solution, but it just doesn't sit well with me. My preferred solution is to use the system stack, but that's where we hit a few dramas... PB's internal calling convention has type casting wedged into it. This makes sense in the VM, since it reduces the number of VM ocodes needed to execute a new scope. But it means that any interface has to be data type aware when pushing or popping. Macros to the rescue !
The solution was to write a set of tiny macros that emulate the behavior of the VM in assembly. Since they're macros, the code is inlined into our exported dll code. So there's no calling a VM function to do the task for us. The up side is it's about as fast as you can get it, the down side is it adds a little extra bloat to the amount of code that's produced around a function and a function call, but personally it's not enough to worry about.
So far, the caller only supports passing primitives such as Integer/Floats & Strings in and out. Multiple returns (inside PB) are supported, but I haven't actually tested to see if they work correctly as yet. Pointer and Array passing isn't currently hooked up..
Some test code:
Function DLL_ExportMe(A,b#,s$)
Dim Dude(100)
Print "Calling Local PB function"
result=TestFunction(A,b#,s$,Dude())
print "result="+str$(result)
EndFunction result
Function TestFunction(A,b#,s$,Dude())
print a
print b#
print s$
result=A+b#
print "array Status"
if getarraystatus(Dude())
print GetArrayElements(dude())
endif
EndFunction result
In this program we see two different types of functions. The first one DLL_ExportMe is our public / exported function. This function is visible to other programs that load this compiled DLL. Since it's a DLL function, it uses the STDCALL calling convention rules. The second function TestFunction (which is called from the first in this example) is a local function to the dll. This function is not visible externally and uses the PlayBasic's calling convention internally, so even if you have a pointer to this code, it's not callable externally.
EDIT #1 - testing multiple returns from functions
This seemed like it initially would work, but there was problem with the translator using a previously defined instruction in an unexpected way. Once that that was picked up it seems to work OK.
Recursion isn't currently supported though.
a=666
b#=123.456
s$="hello"
DLL_ExportMe(A,b#,s$)
Sync
waitkey
Function DLL_ExportMe(A,b#,s$)
Dim Dude(100)
Print "Calling Local PB function"
result=TestFunction(A,b#,s$,Dude())
print "result="+str$(result)
result,result2=TestFunction(A,b#,s$,Dude())
print "Multi returns"
print "result="+str$(result)
print "result2="+str$(result2)
print "------------------------"
EndFunction result
Function TestFunction(A,b#,s$,Dude())
print a
print b#
print s$
result=A+b#
result2=result+1000
print "array Status"
if getarraystatus(Dude())
print GetArrayElements(dude())
endif
EndFunction result,result2
PlayBasic To Dll - Range / ClipRange functions (February 21, 2014)
Added support for the integer Range instruction yesterday, you can't mix types at this point. For expressions that pass Range a mix of floats, it's probably going to be best solved by calling a function (the call size is smaller than my inlined code). Which is something I've been trying to avoid for maths functions where practical.
The ClipRange functions are borderline also, I suspect there's situations where the output code could be simplified more than calling a one size fits all function , like where the range terms are literal. If that occurs, it could output a compare block in it's place. Which would be faster at runtime !
ie.
Result=ClipRange(SomeValue, 100,200)
could become something like this (in BASIC styled logic)
Register=SomeValue
if Register<100 then Register =100
if Register>200 then Register =200
Result =Register
I could just take the easy way and cookie cut everything, but there's plenty of BASIC compilers that already do that..
PlayBasic To Dll - For/Next Problems
It's not all rosey though as one of the problems i've been running into recently, stems from the very nature of the byte code itself. The VM uses self modification in places, which is very handy for removing decisions from certain opcode pairs. But can make translating the logic back something of a pain.
For/Next loops are one of those areas, where the compiler produces all number of combinations at the start the loop sequence, but these opcodes are generic, there's nothing special about them and they appear in other constructs as well, so detecting exacting what's going on isn't turning out to be that easy.. The result of which, is sometimes the translator misses the FOR loop set up code, when that occurs the for/next doesn't work in the exported DLL. I'm hoping there will be some rainbow moment where I can figure out a bullet proof solution, but nothing coming to mind at this point.
There's a few solutions, worse case would be adding bread crumb opcodes to the byte code in order to help any translator(s) process down the line. Another approach would be change how the VM works and use add loop start opcode which would make it easier to translate and possibly at runtime for VM also. I kind of favoring the latter, but we'll see..
PlayBasic To Dll - GUI - Up And Running ! (March 28, 2014)
After a frustrating week of playing hard drive roulette, yesterday i've continued putting all the parts together. The build process is broken into two halves, there's the GUI side and translation engine side. Unlike other helper apps, these are two separate projects, which are combined using a slightly more customized version of the project loader/source merger. This version of the tool lets me wedge the Engine into the GUI easily and rips unwanted blocks of the code from the engine. The pre-processing tool works via looking for tokens inside comments. So the tokens mean nothing to PB, but the pre-processor tool just skims through looking token pairs and rips anything between them.
Example,
; [REMOVECODE]
function Some_Function_Not_USed_By_GUI()
EndFunction
; [/REMOVEDCODE]
So the code (in this case a function) is included when engine when testing separately, but would be ripped when the engine is merged for inclusion into the GUI. I often use such approach to substitute methods in the final source codes for example. Once the engine source is built, I just include it into the GUI project and off we go. The GUI side holds the high level properties and just calls the conversion functions from the engine. To make the engine project output to the GUI rather than the odd print statement we just wrap up our own print function. So depending upon the context, the wrapped function either uses the GUI console or a print to screen.
When running the final program the user has to initially locate their PlayBasic.exe compiler file. This will generally be in your My Programs or Program Files folder it's different between OS's. I can sniff this out only when install in the most common location, but can't if you install to a custom location. To find the file go to 'SETTINGS->CONVERSION' and click the locate button. The dialog is set up to only accept one file. It's just matter of you finding it. Once the location has been set, it remembers this path.
To convert a PB source (PBA file) we go to FILE->CONVERT FILE, this pops a file locate dialog where all you do is select the file. PB2DLL will take over from there, there's is one little issue though and that's what your DLL name is going to be. Unfortunately, you can't create the DLL then rename it later, that won't work as there's naming dependency. So the name has to be given to the DLL up front. There's a few ways around the issue i guess, the DLL name could become the source files name. This is fine for those people that give their files meaningful names, but unfortunately a lot of people don't. One way around the problem would to be search through the compiled byte codes constant table for a constant called "PB2DLL_NAME$" or something similar, another would be to use a DLLNAME_ prefix in a function/psub declarations . Either way the programmer can then simply add the name string to their source code and then not have to enter a name during translation. Which would get old really fast if your constantly editing -> building -> repeat..
So anyway, it's up and running.. just matter of tweaking everything now.
PlayBasic To Dll - Pricing & Back End ( April 01, 2014)
PB2LL is something of a first around here, up until now all the helper tools have been freebies, PB2DLL is a commercial tool. The initial price will be for the early adopter version ! Which will be similar to what PlayBasic retail current costs, so somewhere between $20->$30 US. The final will most likely cost around $50.
One of the most frustrating things when releasing new software is all the secondary stuff that's required. It's not just the software you have to write, it's the ordering infrastructure it requires as well. Will probably go with shareit again, but contemplating just using PayPAL. Much like the DTAB ordering process, the KEY creation side of the ordering process won't be automated. You'll most likely order, then your registration will be sent to you once the order has processed locally. Which is required, since the key builder process is written in PB. Shouldn't be a drama really, as we're unlikely to see more than a few sales.
So today's little chore has been working on the activation system library which is just matter of dragging some of the code around from older libs, was a little painfully setting it up, but the process seems to be working. With other tools i've build pre-processing tools to do the entire process, which is tempting, but again can't really see any great demand..
More Information
For the full picture/ screen shots etc see the PlayBasic To DLL development blog
|
|
|
|
|
| |
|
|
|
|
| |
|
|
|
|
