Blastanova

August 18, 2010

Sambaverse Alpha

Filed under: flash/flex,music video games,projects — admin @ 12:05 am

Finally! I’m at the point where I have a demo-able application. Sambaverse is the first phase of my master plan for Blastanova.

Quite some time ago, I got interested in creating music based Flash games. I created a prototype, and it worked pretty well – but it was severely lacking in one aspect. That aspect was the ability to look ahead in music, to know what’s coming up 5 seconds from now or even 5 milliseconds from now. Or to be able to look back at what happened in the music before. Or to be a little smarter about logical song breaks.

While Flash has some decent realtime audio processing capabilities, realtime just wasn’t good enough – my music games needed mystical, physic powers…to be able to see into the future and know how the musician who composed the piece thinks.

In real terms, I needed a tool that was smart enough to load an MP3, take a good stab at automatically detecting beats, break, loud sections, and different sections of the song – like verses and choruses.

So, I set out to create Sambaverse. A person using this application loads up an MP3. Right away the gears start turning – an audio waveform and song navigator is brought up in a few seconds. You can browse around, and listen to snippets. You can properly visualize your song. That’s not very special though….you can do that in any audio editor. This is why I made some music visualization modes…

using the application
At the upper right of the application is the “Playback Visualization” menu. You can choose one or multiple options here. You have the ability to view the song beats, look where the quiet periods of the song are, where the most intense portions of the song are, and find logical breaks (sections) in the music like where a chorus ends and the bridge begins.

Unfortunately, automatically finding logical breaks in the song was a little tough, and it will never be perfect. I’d describe my break detection as a good start, I have quite a ways to go. This is why I’ve added two drop down options. The first is the “custom song sections”. This mirrors the automatically detected segments, until you drag them around to place them where you want. You can fully customize and refine these segments.

Likewise, I added a song exclamation overlay. Using this mode will place little stars on the timeline. The purpose of these stars is to signify special musical events that occur very rarely. Like when that singer does that one scream in the song “YAHHHHHH!” or there’s a sudden gong sound effect…just something wacky that occurs in the song against the norm.

So you have a few view modes at your disposal. Even more than that, pressing the orange “Analyzation Settings” button will bring up the settings window. Here you can change how quiet your quiet breaks need to be or how long they are in order for them to be identified as quiet breaks.

Theres a few settings for every view mode. I’ll be tweaking these as I go along to see what works best, and probably change names and defaults.

Finally, you can save this data as XML. Actually – I’m not really saving it yet….just posting it to a webpage, and you can save it if you want. Whatever. I know I’m psyched to play around with my new app, tweak some songs, and make some cool games.

Under the Hood

I learned a lotta cool stuff making this. The biggest thing of course was how to manipulate and use audio for my bidding in Flash. I used Pixel Bender to get a huge speed increase when loading the sound initially. That was a lot of fun once I figured out how to get it working.

Although a little annoying at the time, I was using the latest releases of Flex and Swiz. As I’ve been working on this app for a little while, I had to go through and upgrade from Flex Gumbo to Beta 1 to Beta 2 to RC to Release. Stuff changed every time. Likewise, I had an annoying evening transitioning from Swiz 0.64 to 1.0.

But I’m happy I did this. Flex skinning is awesome. Swiz allowed some nice shortcuts and organization in my application. I even tapped into Flash Catalyst for the initial application design. Catalyst seemed a little limiting after the initial graphics dump. After that for quicky and dirty graphics, I just cracked open Illustrator and exported to FXG which I then copied as text into my MXML skins.

future improvements

I’m going to take a break from this application for a bit and focus on some games. I’m sure I’ll tweak as I go along and I find limitations that I need to solve as I’m working on the games. One huge feature I’d like to add when I get back to this is to be able to run a Fast-Fourier Transform on the audio and drop frequencies I don’t care about. This way, I can zero in on the drums or the bass guitar and only analyze those. In rock music especially, the frequencies are all over the place at different tempos and can make it a little hard to find a rhythm.

I’d also like to make it so you can load in a previously saved XML file to get all your work back for a particular song.

Lastly I’d like to get a sample game/animation you can preview as you work to visualize the beats better than the Flash lights I have.

So that’s Sambaverse! Check it out! (but be nice, it’s only an alpha) .

June 23, 2010

A Custom Multi-Selection Spark DropDownList

Filed under: flash/flex — admin @ 11:00 pm

I just had to add a drop down list into my Flex 4 project, but I wanted to be able to multi-select the items, and add some custom logic.  It was sort of a pain in the butt figuring out how to do this, so I thought I’d share!

So, first of all, I created a new class that overrode DropDownList.  I added this “MultiSelectionDropDownList” to my application.  I set the width to 300, set “requireSelection” to false (since we’re allowing checkbox selection which the DDList wouldn’t know about), and set up a data provider with a few items in my list.

All normal everyday Spark DropDownList stuff so far, right?  OK, next, lets create a skin for the list.  The only reason I do this is to remove the “maxHeight” for the list.  I don’t want my list to scroll….because frankly, this isn’t the greatest example and will break when my list items are outside the viewable area.

Next I’ll add an item renderer.  The item renderer will be a normal background, and a HGroup on top of it which shows a label and a checkbox (needs an ID for access).  Pretty simple stuff so far!

Now, I have my multiselection dropdown in my application like so:

<local:MultiSelectionDropDown
		width="300"
		requireSelection="false"
		itemRenderer="MultiSelectItemRenderer"
		skinClass="MultiSelectionDropDownListSkin"
		dataProvider="{menu}"
	/>

At this point we should have a functional drop down showing some items with checkboxes. Of course those checkboxes don’t work yet, and we’ll need to add some logic.

Let’s dive into the class we created that overrides the DropDownList – the “MultiSelectionDropDown”.

The first thing I want to do is in the constructor – I want to add an event listener for when the list opens:

		public function MultiSelectionDropDown()
		{
			super();
			this.addEventListener(DropDownEvent.OPEN, onOpen, false, 0, true);
		}
 
		public function onOpen(event:DropDownEvent):void {
			activateAllCheckBoxes();
		}

So now, whenever someone clicks and opens the list, we’ll be “activating” all of our check boxes. What does this mean? Well, we’re going to loop through all of our list items, and add some event listeners to these checkboxes.

		protected function activateAllCheckBoxes():void {
			for (var c:int = 0; c < dataGroup.numElements; c++) {
				var obj:MultiSelectItemRenderer = dataGroup.getElementAt(c) as MultiSelectItemRenderer;
 
				obj.checkbox.addEventListener(MouseEvent.MOUSE_DOWN, mouseCheckBox, false, 0, true);
				obj.checkbox.addEventListener(MouseEvent.MOUSE_UP, mouseCheckBox, false, 0, true);
				obj.checkbox.addEventListener(Event.CHANGE, changeCheckBoxSelection, false, 0, true); 
}
 
		protected function mouseCheckBox(event:Event):void {
			event.stopImmediatePropagation();
		}

OK, so we’ve added some event listeners. I’ll get to the actual code to change the checkbox selection in a sec – but check out “mouseCheckBox”. Here we’re grabbing the event, and stopping it’s propagation. This means that the buck stops here when you click a check box. The event won’t make it’s way down to your item, and won’t select your item in the list and close the dropdown. Great! Cause, we want to make multiple selections without having to bother with opening the list over and over again.

Now, lets focus on the actual clicking of the checkbox:

		protected function changeCheckBoxSelection(event:Event):void {
			currentlySelectedCheckBoxes = selectedCheckboxes;
 
			// turn on multi-view mode
			if (event.currentTarget.selected == true ) {
				selectedIndex = -1;
				labelDisplay.text = "multiple selections";
			}
 
			dispatchEvent(new Event("selectionChange"));
		}
 
		protected function get selectedCheckboxes():Array {
			var returnList:Array = new Array();
			for (var c:int = 0; c < dataGroup.numElements; c++) {
				var obj:MultiSelectItemRenderer = dataGroup.getElementAt(c) as MultiSelectItemRenderer;
				if (obj && obj.checkbox.selected) {
					returnList.push(obj.data);
				}
			}
			return returnList;
		}

So here everytime I click a checkbox, if I turned it on (or checked it), I change the label display to indicate that I’m making multiple selections and set the selected item index to -1. This will make the list indicate to the user that nothing is selected, but we’re in “multiple selection mode”.

Additionally, I’m saving the array of my “currentlySelectedCheckBoxes”. I get this array by looping through my items again, and looking at the checkbox testing if it’s selected.

Once this is saved, I dispatch a “selectionChange” event. What’s this? Well, I made it up, and I’m only using it internally in this class. Who’s listening? This guy….

[Bindable(event="selectionChange")]
	public function get selectedViews():Array {
		var multiSelect:Array = selectedCheckboxes;
		if (multiSelect.length > 0) {
			return multiSelect;
		} else {
			return [selectedItem];
		}
	}

This is a bindable function to provided external access to the array of selected items in the list, whether they are selected through the checkboxes, or just selected in non-multi-selection mode just by clicking on the list item.

Let’s also improve the experience by using our saved array of “currentlySelectedCheckBoxes” and using that to populate the list when the list opens. You see, when our list closes, our items sorta….disappear. It’s sort of annoying, but pretty nice for memory management. Anyway as we go through and add our event listeners when the list opens, we can check if the checkbox is in our saved list, and force it to check itself:

protected function activateAllCheckBoxes():void {
	for (var c:int = 0; c < dataGroup.numElements; c++) {
		var obj:MultiSelectItemRenderer = dataGroup.getElementAt(c) as MultiSelectItemRenderer;
			if (obj) {
				// no check box here, this playback mode only is a solo view
				if (dataProvider.getItemAt(c) == "Select This Item Only" ) {
					obj.checkbox.visible = false;
				}
 
				// find and check of previously checked boxes
				if ( currentlySelectedCheckBoxes.indexOf(dataProvider.getItemAt(c)) != -1 ) {
					obj.checkbox.selected = true;
				}
 
				obj.checkbox.addEventListener(MouseEvent.MOUSE_DOWN, mouseCheckBox, false, 0, true);
				obj.checkbox.addEventListener(MouseEvent.MOUSE_UP, mouseCheckBox, false, 0, true);
				obj.checkbox.addEventListener(Event.CHANGE, changeCheckBoxSelection, false, 0, true);
			}
		}
	}

But that’s not all! I’ve added a “solo-selection” click in my dropdown. Clicking this “solo-selection” will clear out all checkmarks and start from scratch – exiting the multiple selection mode. You might notice that I snuck in a line to hide the checkbox if the list item is that “Select this item only” item. That’s because it’s SOLO selection and it doesn’t get a checkbox so we can select it and other items.

override protected function item_mouseDownHandler(event:MouseEvent):void {
	// launch solo view mode when clicking "playback only"
	if ( event.currentTarget.data == "Select This Item Only" ) {
		deselectAllCheckBoxes();
	}
 
	if (selectedCheckboxes.length == 0) {
		super.item_mouseDownHandler(event);	
		dispatchEvent(new Event("selectionChange"));
	} else {
		closeDropDown(false);
	}
}
 
protected function deselectAllCheckBoxes():void {
	currentlySelectedCheckBoxes = [];
	for (var c:int = 0; c < dataGroup.numElements; c++) {
		var obj:MultiSelectItemRenderer = dataGroup.getElementAt(c) as MultiSelectItemRenderer;
		if (obj) {
			obj.checkbox.selected = false;
		}
	}
}

To accomplish this, I overrode the “item_mouseDownHandler”. Inside here, I said, “if you’re that solo button, let’s deselect all the checkboxes”. To deselect, I simply went through each item in the datagroup, accessed it’s checkbox and turned off the selected property.

I also used this function to determine if we were in multiple selection mode – if so, then close the dropdown list without committing and values. If I wasn’t in multiple selection mode, then just do what the list does normally – except dispatch that change event so our custom bindable property updates any outside listeners that our selection has changed (even though it’s just a single item now in the array).

Finally the very last thing I did….

protected override function dropDownController_closeHandler(event:DropDownEvent):void
{
	if (currentlySelectedCheckBoxes.length > 0) {
		// if checkboxes are selected prevent the default behavior,
		// which is to set a selection index
		event.preventDefault();
	}
	super.dropDownController_closeHandler(event);
}

When closing the list, normal behavior is to commit a selection. I actually don’t know the details of what it was doing – but it was grabbing a selection from somewhere and producing some funky results I didn’t want if I had multiple checkboxes selected. It would update the label property and selected item with the last thing I selected in my normal list (even if it was a while ago). It’s like the list was remembering something from somewhere and competing with me! Not good.

So, I overrode the close handler. If I had multiple things selected, I stole the event, and called preventDefault(). In the DropDownListBase class, it sees this default is prevented and doesn’t try to commit whatever the heck it’s trying to commit. It’s just lets it be, and closes the list and doesn’t change anything.

That’s it! You can see the list in action, and right click to view the source here:

Multi-Select DropDown

It’s such a tiny thing, but ended up being sort of a pain to figure out how to do – so hopefully I save someone else the pain. Oh and one more thing…I don’t know that I’m necessarily advocating this kind of component yet. Might be kinda complicated in terms of useability. I’ll find out as I use it in my project and tweak it.

May 23, 2010

Music Visualization and Papervision

Filed under: flash/flex,music video games,personal — admin @ 8:46 pm

After watching Simon Free’s Papervision demo at NCDevCon today, I just had to post an old demo I did when I first got my hands on Papervision a couple of years ago. It’s a really cool project (Papervision that is), and I hope to play with it again real soon.
In this demo, you drive around a submarine through the sea, and various fish come into your view. But then a James Brown song comes on, and all the fish start dancing to the music. Different fish listen to different frequencies. It’s fun, but driving the submarine is very hard, because you can lose it off screen….

Anyway – spacebar to make the submarine go, and arrow keys to make it turn (arrow keys are relative to which way the sub is facing)

Papervision Beats (with Dancing Fish)

(by the way, thanks to Troy and Iris Stratton for getting me started with PV3D and Blender!)

May 22, 2010

Do You Belieeeeve in Flash Autotuning

Filed under: flash/flex,music video games,projects — admin @ 2:52 am

No, I didn’t accomplish the mythical Flash autotune in time for my “Audio Manipulation in Flash and Flex” presentation for NCDevCon this Sunday.  But I don’t see why it’s not possible in the least

I didn’t accomplish it, not because of Flash, but because of my lack of digital audio experience.

About a month ago, I was thinking what effect I could dazzle my attendees with, and ONE thought popped into my head:  AUTOTUNE!!!  This led me down the rabbit hole of crazy amounts of math, signal processing theory, code optimization, and more.  I even started reading a free PDF 600 page book by Stephen Smith http://www.dspguide.com/.

I’ve learned about what Fourier Transforms REALLY are, that they are really useful beyond getting frequency data, how to do low pass filters, and tons of other stuff.

I really didn’t think it would be this involved.  This is like an entire field of expertise.  I honestly thought I could steal some algorithm online and be good to go.  Nope, I’m swimming with using the FFT, then back with an iFFT, convolving signals together, and all this crazy stuff.  It’s pretty awesome though, the things you can do and learn from audio signals, and any signal in general.  I’ll definitely be finishing that 600 page book (probably 3-4 times over so I understand it).

Anyway, I was able to accomplish pitch shifting in a couple different ways, and riding a voice on top of another tone (which sounds really close to autotune if you could just get rid of that damn tone!)

I didn’t even think that one of the things that autotuning did was to detect the frequency of a sung note, and step it up or down to the correct frequency.  I thought at the beginning that when T-Pain did his thing, he just sung whatever, and the software would push the voice to whatever the producer wanted.

With Flash, I’ve seen the Audio Processing Library for Flash detect notes in a sound, and as I’ve said, I can now pitch shift!  In truth, the real Autotune by Antares Audio Technologies is said to use a “phase vocoder”, which I’m still not up on my theory enough to know what it is.  It’s probably a combination of smart pitch-shifting coupled with a flange like effect to go all robot sounding.

I finally downloaded 10.1 and got my microphone working – I recorded in from the microphone, and played back via the sound buffer (all through the sample data event).  I pitchshifted first by just speeding up the tempo so things got all high pitched and fast.  But then I grabbed a PitchShifting class by Stephan M. Bernsee  that was ported to Actionscript from C# by Arnaud Gatouillat.  Using this was VERY processor intensive.  In fact, in debug mode, my entire computer was overheating, and it was a crapshoot whether the sound would actually come out right.  And then of course the Windows sound buffer kept doing weird snapping/popping noises every so often until I restarted.  However, running as a release build seems to work just fine.  I now know why Andre Michelle’s AudioTool has a warning against using the debug player.

But all in all, it was a great learning experience.  I’m embedding the demo I’ll be showing at NCDevCon on Sunday.  And of course I’ll keep learning to become a DSP master, and someday get my Autotune working (hopefully someone beats me to it, and shares the code).

Autotune Attempt – You Need Flash 10.1 and a Microphone

April 8, 2010

Custom Events in Actionscript 3

Filed under: flash/flex — Tags: , , , , — admin @ 1:04 am

A friend of mine recently tweeted that found that he could create custom events for his project, but he found it rather mysterious – and he didn’t really understand what he was doing even though he followed the instructions and was able to make them work in his project.

Since I use nothing but custom events in my projects, I thought they’d be a good blog post subject.

So…..custom events.  Why do you want them?

Well, first lets think about the everyday Flash events. I use flash.events.Event and flash.events.MouseEvent all the time. There are the obvious uses, like when you click on something – you listen for the “click” event type by saying addEventListener(“click”, onClick).

But, how do you know it’s “click”? It could be anything. I hate looking at the docs for every little thing. So, it’s much easier to type “MouseEvent.” into my code editor and watch it auto suggest stuff for me. You can look through and see MOUSE_DOWN, MOUSE_UP, CLICK, etc, all as options.

So I end up with addEventListener(MouseEvent.CLICK, onClick). It’s easy, there’s no chance I mistyped it because I didn’t type it – my code editor did.

But then you might be moving on to creating your own events. Say you’re loading a file. Whether the file loads, fails, the user cancels, you want to know when it’s done – so you might dispatchEvent(new Event(Event.Complete)) to let other parts of your project know that the operation is complete regardless of how it completes, just so those other parts know the file loading is done and they can move on with whatever they’re doing. Event.COMPLETE is handy for whenever you want to wire up completion events – just like Event.CLOSE is great for when you have stuff that’s closing and Event.INIT is great when you have stuff that you’re finished initializing.

What about other types of events? What about events that are specific to your project? For example, say I’m making a game. In the game, I have a rubber ball that the user can interact with. The user can bounce the ball, roll the ball, throw the ball, and spin the ball. The logistics of the user interaction to the ball doesn’t matter here, we’re just worried about the events that get dispatched.

Let’s name our events:

1. “bounceBall”
2. “rollBall”
3. “throwBall”
4. “spinBall”

Now, I could simply just use a normal Flash event. My ball dispatches events in the fashion of “dispatchEvent(new Event(“bounceBall”));”

…and I listen to my ball bouncing by adding an event listener: “ball.addEventListener(“bounceBall”, onBounceBall);”

That would work OK, but I’d have to keep looking up what I named the different events, and make sure they match in all parts of my project. God forbid I make a typo – I could be stuck for hours with a “bounceeBall” event not knowing that I accidentally typed an extra “e” in my dispatcher.

No – I like to name my events in static constants, and put them inside the event. That way I can start typing “BallEvent…”, and my code editor will autofill the rest.

So the first step is to make a BallEvent by overriding the Flash Event class:

package events
{
	import flash.events.Event;
 
	public class BallEvent extends Event
	{
		public static const BOUNCE_BALL:String = "bounceBall";
		public static const THROW_BALL:String = "throwBall";
		public static const ROLL_BALL:String = "rollBall";
		public static const SPIN_BALL:String = "spinBall";
 
		public function BallEvent(type:String, bubbles:Boolean=false, cancelable:Boolean=false)
		{
			super(type, bubbles, cancelable);
		}
	}

So I have a nice little BallEvent class. Now there’s no chance of typos. I can dispatchEvent(new BallEvent(BallEvent.ROLL_BALL)) and addEventListener(BallEvent.ROLL_BALL, onRollBall)); and if I DO make a typo, my code compiler will complain and tell me where I went wrong before my SWF ever gets created.

One thing worth mentioning is the “public static const”. If you aren’t aware of this syntax – a static is a variable that is accessed as a singleton. That is, you don’t need to create a new BallEvent object in memory to use it – just say BallEvent.ROLL_BALL, and you’ll get the value: “rollBall”. Also, a “const” is a constant value – it’s something that never changes and you hardcode it ahead of time and don’t set it at runtime.

So that’s one advantage of custom events – and a good one. Things are readable and typos don’t exist. Furthermore, things can be grouped pretty well. My BallEvent class can have all my BallEvents whereas my FrisbeeEvent class can have all my Frisbee events…. it just leads to good organized code.

Now that’s all well and good – but maybe you said to yourself “Self, it’s great I know that my Ball is bouncing, but what time did it bounce, and at what angle, and at what velocity?

Well, I could receive the Ball bounce event, and say:
var bounceAngle = ball.bounceAngle;

But what good is that? The angle of the bounce is a snapshot of a moment in time. Maybe the angle has changed in the time between when I got the event and when I’m asking the ball what the angle is. All these things can change – ESPECIALLY when you change them yourself. What if you had special code to increase the velocity of the ball when it bounces off something? Well, guess what? You could increase the velocity on the ball bounce event, but then later on in your code, you could try to get the velocity. These order of event things can get confusing. Are you getting the velocity before the you changed it? Or is it after you changed it? It can get downright undependable and muddled.

However, if you included these values in your event, you can insure that you are grabbing these properties at the moment in time that they happened, whenever you want rather than relying on guesswork.

So let’s add a couple properties to the BallEvent. I want to track velocity, time, and angle.

public static const SPIN_BALL:String = “spinBall”;

                public var time:int;
                public var velocity:int;
                public var angle:int;
 
		public function BallEvent(type:String, eventTime:int, eventVelocity:int, eventAngle:int, bubbles:Boolean=false, cancelable:Boolean=false)
		{
                        time = eventTime;
                        velocity = eventVelocity;
                        angle = eventAngle;
			super(type, bubbles, cancelable);
		}
	}

So here, I just added the properties I want to the event. When I dispatch the event, I pass them in: dispatchEvent(new BallEvent(BallEvent.ROLL_BALL, 50, 21, 90));

And then when I get receive the event, I can get all the properties I need!

public function onBallRoll(event:BallEvent):void {
var myAngle:int = event.angle;
}

That’s the basics of the custom event – but there are some best practices that folks like to follow….

The first one is to implement a “clone” method. Let’s say that you listened and caught the event, but you really just want to pass it on to another class by dispatching the event from the class that caught it. Well you can call event.clone(), and you’ll get a copied event. Unfortunately, those extra properties you added won’t be cloned. So let’s look at a sample overridden clone method:

	public override function clone():Event {
		var event:BallEvent = new BallEvent(this.type, this.time, this.velocity, this.angle, this.bubbles, this.cancellable);
		return event;
	}

So here you’re actually creating a new BallEvent in this method and returning it when you clone. You’re using your public properties that you set when you originally made the event, and just rolling them into this new event. The only thing a little unintuitive here is that the return type is a normal Flash Event. Well.. it needs to be, because you’re overriding the Flash Event’s clone function (Flash will complain otherwise), but don’t worry – when you receive the event it will be cast as the BallEvent with all the properties you need.

For example, in:

public function onBallRoll(event:BallEvent):void

The event, if it originally started off as a BallEvent (even if it got casted to Event through cloning or otherwise), will still come through here as a proper BallEvent.

Honestly, sometimes I get lazy and don’t implement a clone event. Then several weeks later I need to clone the event and I curse my laziness. But sometimes that never happens. So it’s up to you if you think you need it.

Another “pro-tip” is to use getters and setters. Remember the concept of your properties, and how they represent a snapshot in time of the Ball? Well, some people feel that this isn’t good enough. Say I’m an evil programmer, intent to sabotage you and your events. I could listen for your event, program a line that says

event.time = 9999;

….and then clone and dispatch your event. You’d never catch on to my evil attempts. I’ve changed a property on the event – and then I passed the event on like nothing happened. It no longer represents a true snapshot in time – more like a doctored snapshot in time.

So, some folks try to stop this. They create getters but no setters. That way if an external class tries to change the property, it will cause your compiler to throw errors, thus saving you from malicious programming.

Let’s take a peek at a prim and proper finished BallEvent:

package events
{
	import flash.events.Event;
 
	public class BallEvent extends Event
	{
		public static const BOUNCE_BALL:String = "bounceBall";
		public static const THROW_BALL:String = "throwBall";
		public static const ROLL_BALL:String = "rollBall";
		public static const SPIN_BALL:String = "spinBall";
 
                protected var _time:int;
                protected var _velocity:int;
                protected var _angle:int;
 
                public function get time():int {
                    return _time;
                }
 
                public function get velocity():int {
                    return _velocity;
                }
 
                public function get angle():int {
                    return _angle;
                }
		public function BallEvent(type:String, eventTime:int, eventVelocity:int, eventAngle:int, bubbles:Boolean=false, cancelable:Boolean=false)
		{
                        _time = eventTime;
                        _velocity = eventVelocity;
                        _angle = eventAngle;
			super(type, bubbles, cancelable);
		}
 
	        public override function clone():Event {
		     var event:BallEvent = new BallEvent(this.type, _time, _velocity, _angle, _bubbles, _cancellable);
		     return event;
	        }
	}

Well, OK maybe if I was being all proper, I’d throw some ASDoc or some other type of comments in there – but you get the picture.

But my last point here is that I’ve set all my variables as protected (and I usually put an underscore in front of my protected variables because of a habit I have). The protected variables can’t be get or set except by the BallEvent class or anything that extends it (like a RubberBallEvent class or a BaseBallEvent class). So these properties are completely locked in and can’t be changed by other classes. But I make them accessible through my getters.

There are other little tricks I sometimes do too. Like in a current project I have a dashboard full of buttons. When I click my dashboard, I dispatch a Dashboard.CLICK event type. But then the actual button clicked is a property called ID, or maybe it’s a reference to the actual button.

This saves me a step. Instead of adding an event listener for each and every button in my dashboard, I can simply add one event listener to my dashboard for clicking an event. And then I can have logic in my event handler for deciding what to do based on what event was clicked.

Events are usually pretty simple too, but sometimes you can make them more complex. Like say you could dispatch an event with a time property, but then have several methods to format that time property however you like. Maybe your time is originally a huge number like 22531245665.

Getting this time might be useless to display to normal people. Maybe you want to include a formattedTime function in your event. Instead of getting event.time, you could get event.formattedTime and get “April 17, 2010 2:03PM”.

So that’s how to do custom events! Hope it helps.

March 24, 2010

Intro to Pixel Bender

Filed under: flash/flex,music video games — Tags: , , , — admin @ 11:37 pm

So this is my intro to using Pixel Bender – if you don’t know what it is or why to use it check out the documentation or my first post.

Actually the documentation is a good place to start.  Go to http://www.adobe.com/devnet/pixelbender/ before you begin.  Go on….I’ll wait…

Once there you’ll want to download the Pixel Bender Toolkit.  It’s a simple and light program – not subject to the lengthy installs of other Adobe software.  Don’t bother downloading the PDF documentation, there are links in the help menu to these documents once you run the toolkit.

The best part of the documentation – which actually turns into the most depressing part once you learn it, is that you ignore around half of it if you’re developing Flash shaders.  Most of the more advanced functionality only applies to Photoshop and After Effects shaders.

So crack open the toolkit!  What to do now?  Well first, go to the file menu and choose “new kernel”.  Kernels are basically the “programs” you’re creating that compile to shaders.  A new kernel will look like this:

kernel NewFilter
&lt;   namespace : "Your Namespace";     vendor : "Your Vendor";     version : 1;     description : "your description"; &gt;
{
    input image4 src;
    output pixel4 dst;
 
    void
    evaluatePixel()
    {
        dst = sampleNearest(src,outCoord());
    }
}

Don’t worry about that top part – it’s just noting the author of the script.

So the first thing to worry about are the two variables at the top. There’s “image4 src” and “pixel4 dst”. You might guess that you’re defining a source image and a destination image. But what’s up with the funny syntax?

Well, first of all, Pixel Bender is one of those languages where the data type is in front of the variable. So you have a variable “src” of type image4, and a variable “dst” of type pixel4. Image and pixel datatypes might make sense, but the “4″ is what threw me off at first, but don’t worry it makes sense.

PxB mainly deals in floating point numbers. And no automatic type conversion! Doing float var = 2 is no good, but doing float var = 2.0 is OK. There are basically 4 types of floating point numbers: float, float2, float3, and float4. A float4 is basically an array of 4 floating point numbers. Example: float4 myfloat = float4(2.0, 2.0, 2.0, 2.0);

It only goes up to 4. Once you realize that the main point of PxB is to manipulate pixels, you being to see why. Red + Blue + Green + Alpha = 4 channels and an array of 4 floating point numbers.

I’ve found that I can use pixels and floats interchangeably (maybe I’m wrong). Images are reserved for an entire image comprised of pixel 4′s/float 4′s. Pixel Bender also supports integers and booleans (each with 4 or less values).

OK that was my rant on variables. Lets move onto “evaluatePixels”. This is THE method that everything PxB revolves around. In fact, in Flash, you can’t even create other methods in your kernel (PS and AE allow this though).

Every PxB kernel is designed to do one thing and one thing only. Take in a source image, go pixel by pixel, and create an output image pixel by pixel from the “evaluatePixels” method.

That’s easy enough to understand – but what about that wonky syntax they start you off with?

dst = sampleNearest(src,outCoord());

So, lets work from the inner to the outer. Starting with outCoord(). This method gets the current coordinates that pixel bender is analyzing at that moment. Hint: it’s a float2, containing both X and Y values.

That was the easy part – the hard part is “sampleNearest()”. For us Flash folks, this introduces you to the wholly confusing notion of sampling on half pixels and pixel ratios and other such nonsense. But then you realize that its Flash, and all pixels are square, and sampling a pixel samples the entirety of the pixel. At this point you realize that sampleNearest is just how PxB works – but it’s entirely unnecessary for Flash.

So in other image editing software (and apparently this holds true especially for video), pixels don’t have to be square. Pixels can have a different height and width, giving them an aspect ratio, which you can actually check for in PxB.

But then there’s PxB…it will scan each pixel in the image as IF THEY WERE square. So you end up with coordinates that could be x:4.56, y:1.567. When you do “sampleNearest”, you’re sampling the nearest pixel to these fractional values to end up with nice locked-in PxB world coordinates like x:4, y:2. You can also call “sampleLinear” which takes the average of the surrounding pixels when you ask for something on a half pixel.

Betcha feel smart now, don’t you? Well forget everything you just learned. If you are doing things in Flash, all pixels are square, and all pixels match to the PxB world coordinate system perfectly. So “sampleNearest” is just something you have to do to get the red, green, blue, and alpha values of the pixel.

So – in the end…you’re just taking the pixel you’ve come to, evaluating the 4 channels, and dumping those right back into the destination pixels. In other words, you’re doing nothing.

At this point though, it becomes easy to start manipulating an image. Go to the file menu again, and load an image. PxB has some sample ones to use, like this one:

Now change dst = sampleNearest(src,outCoord()); to:

dst = sampleNearest(src,outCoord()) * float4(0.25, 1.0, 1.0, 1.0);

Now click “run”

Congratulations! You just went into every pixel and turned the red down to 25%.

How bout a weird cross-hatch type effect?

dst = sampleNearest(src,outCoord()) * float4(sin(outCoord()[0] * 4.0), cos(outCoord()[1] * 4.0), sin(outCoord()[0] * 4.0), 1.0);

Play around, try different things. If you break anything, Pixel Bender will give you red error messages of varying usefulness on the right side.

The hardest thing to get used to is always typing numbers with decimals and usually performing operations not with one set of numbers but with a set of 4. If you run into any trouble, keep asking yourself these two questions:

  1. Am I performing a mathematical operation on two different data types?  Float2 * Float4 = Error!
  2. Am I performing a mathematical operation on a float using a number with no “point zero” on the end?  Float * 2 = Error!  Float * 2.0 = Good!

So that’s the basics of PxB!  You can manipulate surrounding pixels if you like by performing operations on surrounding pixels.  Just add or subtract X and/or Y to your outCoord(), and sample that pixel.  Combine and average surrounding pixels to get a blur effect for example.

Here’s an example of taking a big image, and downsampling the image to a tiny corner in the upper left of the destination:

        float4 colorAccumulator = float4(0.0,0.0,0.0,0.0);
        float4 avg;
        colorAccumulator += sampleNearest(src, outCoord() * float2(9.0, 9.0) + float2(-1.0, -1.0));
        colorAccumulator += sampleNearest(src, outCoord() * float2(9.0, 9.0) + float2(0.0, -1.0));
        colorAccumulator += sampleNearest(src, outCoord() * float2(9.0, 9.0) + float2(1.0, -1.0));
        colorAccumulator += sampleNearest(src, outCoord() * float2(9.0, 9.0) + float2(-1.0, 0.0));
        colorAccumulator += sampleNearest(src, outCoord() * float2(9.0, 9.0));
        colorAccumulator += sampleNearest(src, outCoord() * float2(9.0, 9.0) + float2(1.0, 0.0));
        colorAccumulator += sampleNearest(src, outCoord() * float2(9.0, 9.0) + float2(-1.0, 1.0));
        colorAccumulator += sampleNearest(src, outCoord() * float2(9.0, 9.0) + float2(0.0, 1.0));
        colorAccumulator += sampleNearest(src, outCoord() * float2(9.0, 9.0) + float2(1.0, 1.0));
 
        dst = colorAccumulator/9.0;

My next post will be about taking Pixel Bender and using it for non-image data processing. Stay tuned!

March 19, 2010

Pixel Bending for Speed in Flash and Flex

Filed under: flash/flex,music video games — admin @ 1:17 am

Lately, as I’ve been progressing with a Flex 4 based application which is largely an audio visualizer, I’ve felt the pain of slow response times from my UI.

The slow response is due to the fact that I’m loading a 3 minute MP3, extracting the audio to a byte array, and then processing the data in that byte array.  Consider a 180 second song with 44,100 samples per second.  That’s 7.9 million numbers to process as fast as I can.

This has caused me to do some complicated things.  The best of my efforts entailed processing the audio in segments on every enterframe handler.  I’d tell flash to process a reasonable amount of data, stop, and then process more the next round until everything was finished.  If I remember correctly, this took around 20-25 seconds or so, and STILL made my UI very sluggish as I was processing the data.  And of course this was in my development environment – in real life, a user would be waiting even more time while an MP3 loads.

So, I had two basic problems.  The first was that data takes too long to process.  I can hide this potentially with a good user experience – after all people expect that loading a large file can take some time.  But this leads to my second problem – Actionscript runs on one thread.  Only one thing can happen at a time.  If I’m processing audio, I can’t be updating/refreshing my UI.  And the whole application becomes unresponsive.  The more responsive I make my UI, the less data I process in a frame – but this just makes the whole darn thing take longer to process. Worse yet, if I process the whole thing in one go, Flash can timeout from inactivity at 15 seconds.

Basically it boils down to running it all at once, and potentially doing this:

or this….

Fortunately, there are two such Flash technologies that had the promise of helping me out.  First there’s Alchemy.  Alchemy is an Adobe research project that compiles C++ code to a Flash SWC.  Supposedly it can run code up to 10x faster than code compiled with Actionscript.

But, I haven’t touched C++ in a while, and still had the problem of an unresponsive UI for the time it takes to process the audio (whatever that time is).  It was tempting, but I had my sites set on trying Pixel Bender.

Pixel Bender is a cross-product Adobe technology.  It runs in Flash, Photoshop, and After Effects.  Basically it allows developers to write their own image filters.  In Flash, you can apply this image filter to images, animations, video, components…..and well anything that displays on the stage.

The best part?  It can run in different threads, on different CPU cores, and on your GPU.  Well, actually, strike that last part….you can’t run it in Flash on your GPU, but Photoshop and After Effects are cool.

This means that you can run a Pixel Bender shader on an image, and your UI doesn’t slow down.  Maybe you wrote something insanely complicated, well…then your PxB shader will suffer performance, but your UI won’t!

It doesn’t stop there – you don’t even have to have an image as your…erm…source image.  Yes, PxB assumes red, green, blue, and alpha channels.  But you can easily lie to PxB, and have it assume that your custom data is RGBA data.

This brings me full circle back to my problem.  I have something that supposedly processes data quickly, and on a different thread.  Hooray!  In fact, there’s a few projects going on that use PxB for audio processing already.

Come to the March RDAUG meeting on Tuesday to find out more!  I’ll be presenting, and quickly talking about what I covered here, but also diving into the nitty gritty.  You know….how to actually use this stuff in your work.

I’ll also be following up this blog post with a second part next week covering what I went over in my presentation.

February 9, 2010

Vacation from Flash

Filed under: flash/flex — admin @ 9:14 pm

I just came back from a long weekend trip to Orlando, Florida.  Just prior to that I had been working to launch this website – complete with a jQuery driven photo album, Joomla based CMS, and WordPress based blog.

This past weekend, waiting in the long Disney World lines, I had plenty of time to read the tweets coming in as the Adobe development teams and the Flash developers I know responded to Flash bashing and HTML 5 superiority in the wake of the Apple iPad announcement and the impending release of Flash Player 10.1, which Adobe is pushing for mobile devices.

I’m not going to add anything to this argument, but my current opinion stands that everything has it’s time and place.  People use Flash horribly just like they’ll use Javascript and HTML 5 horribly, and likewise each will produce some great things.

While most folks are comparing the end result when you create with the different technologies, what was eye opening in creating my website is the comparison in the development process between Flash, Flex, HTML, CSS, and Javascript.

Javascript and Actionscript are the most obvious comparisons.  Both have pretty much the same syntaxes.  Both are ECMA based, however Javascript has a few less features, that to me….can slow development and can lead to poorly written and badly maintained code.  Please note that I said CAN slow and CAN lead to!

Let’s go back in time a little.  Flash 5 and 6 were great at the time,  but didn’t encourage any good programming standards.  Basically you’d write scripts on each frame of your animation.  You could also (and many people did) go into each object and animation and write scripts on those as well.  In the end, you could have a huge unorganized mess which was impossible to comprehend or troubleshoot.

If Flash was your first development environment, over time, you’d start to see that you’ve created big unorganized messes and start to think about how to better maintain your projects.  However, there was absolutely nothing in Flash that encouraged this behavior.

Flash 6 and Actionscript 2 started rolling out with new enhancements to the ECMA standards.  This included the ability to organize your code in classes, and really rounded out an object oriented approach to development.  You still weren’t forced to acknowledge OO practices in your work, but the option was definitely there and some of the more experienced programmers could utilize them.

With Flash 9 and Actionscript 3, developers who tried to do anything substantial were forced to start acknowledging OO practices.  This approach led to Adobe facing some complaints from the casual Flash developer, but also led to the larger community adopting some good (or better) practices around object oriented development.

I think this very much relates to the state of Javascript today.  Someone coming from a more robust language like Java or C++ could pick up both Flash 5 or Javascript today.  They could absolutely introduce a code structure to mimic  an organized object-oriented approach.  However, the reason they can do this is because they have the experience to know how to better organize code, and they know right off the bat what features are missing, and can concoct ways around them.

Contrast this to someone just learning Javascript or Flash 5 as their first development language.  They just wouldn’t know any better than to create lots of inline code, not realizing that classes or methods even exist.  As this person gets more and more experience under their belts, they would see how their approach lacks maintainability and strive for something better.  In the interim however, this leads to horrible, mangled, and unmaintainable scripts.

Actionscript has increased by leaps and bounds over the years, but Javascript still has not.  Great code can and will be developed in each environment – but I just don’t think that Javascript encourages any good practices.

Keep in mind that I’m well aware that the exact same thing can be said about Java/C++ versus Actionscript with Actionscript on the low end of the totem pole here.

The other thing to point out is variable typing.  Having to call a variable a string, number, etc when first declaring it seemed like a nuisance, but turned out to speed up development quite a bit.  This is an important thing that Javascript lacks.

Someone, a year back, turned me onto Aptana for use when developing with HTML/CSS/Javascript.  Aptana is fabulous.  I get to use my experience with Eclipse and Flex/Flash, and just switch to something that looks exactly the same to develop my non-Flash web pages with.  Code hinting is huge for me.  I don’t constantly have to look up syntax, and can start typing and have Aptana suggest what to do next.  This is incredibly helpful for CSS, since I don’t remember all the properties and need constant help.

With Javascript, on the other hand, code hinting is hindered a great deal.  If I don’t declare a variable type, Aptana can’t suggest what properties and methods I can use on an object very accurately.  It actually does a great job guessing (especially with jQuery), but it can only go so far if it doesn’t know what kinds of objects you’re working with.

I suppose you could call CSS a win on the non-Flash side.  CSS, to me, is simply a joy to work with given the right tools.  Aptana, as I said before, does well with code-hinting for CSS.  As a Flex developer who knew enough CSS to style text in HTML and style my Flex components, really getting into CSS was a little daunting.  It’s a lot more powerful and robust than I would have thought.   Flex doesn’t give you the ability to control layout and positioning with CSS, nor does Flex 3 allow inheritance and chaining styles together (Flex 4 does though!).  It can also be a little confusing in Flex to know which properties to set for what you want to do.

All in all, I think I enjoy style and layout in HTML/CSS more.  I’m still a newbie, but it’s my shiny new toy.  Unfortunately, my shiny new toy is very tarnished by having to maintain compatibility in Internet Explorer.  I recently tried and failed to use a glow effect for some text that looked great in Firefox and Chrome, but looked awful because of IE’s clear type.  And then of course, there’s no real practical way to use vector graphics (IE doesn’t natively support them).

For pure programming power, I’ll choose Flex and Flash any day of the week.

I look forward to getting more experience with Javascript and CSS.  It’ll definitely help when I’m evaluating a project.  I’m sure I lean too far on the Flash side right now, and I look forward to being able to better approach a project from any perspective.

July 14, 2009

Composing Music with the Flash ByteArray

Filed under: flash/flex,music video games — admin @ 11:43 pm

Demo here, view source for code

(view source for code, and who knows what bugs there are)

I’m getting into unfamiliar territory these days with exploring sound in Flash.

Flash 10 gives you the ability to take a sound object, and extract the entire thing into the raw data….a byte array.  How do you interpret this data? How can you process it into something meaningful?  Well I don’t know.

What I do know is that to get a firmer grasp on how I can utilize this, I need to understand what the sound object is at a basic level.  Probably the worst way to understand it is to take an entire song (mp3) file and let my eyes glaze over at the stream of numbers coming from the sound.extract feature.

No – the best way, I thought, would be to compose my own notes and chords, and work up from there.  To do this, I needed to understand what a note is, and what a sound is at a basic level.

My first thought was to picture sound as the nifty looking sound visualization that comes with music players these days…you know, those dancing bars:

Turns out that this is the worst way to imagine sound for this purpose.  My first question was….”OK, how do I get the value of the low frequency?”, “how about something in the midrange?”, “what about the highend?”.

Yes you can imagine sound this way, but only after processing your sound data with a Fourier Transform.  If you know how to do this, stop reading this right now, cause you’re way smarter than I am at this point in my sound exploration.

The BEST way to imagine sound is to picture a sine wave:

If you look at how tall the wave is, that is how loud the sound is, or the amplitude.   How close the peaks and valleys are in this picture is the frequency.

Frequency, at this very basic level can’t be thought of as high pitched, low pitched, etc – it’s only how far apart these peaks and valleys are.  Now think of this as a side-scrolling wave that goes on forever.  If you were to scroll at a constant speed, each peak would hit that vertical center line at a constant rate.  This rate would be the frequency.  If it goes faster, the frequency is higher – slower, the frequency is lower.  And, of course, this directly relates to how you hear it.  The more peaks and valleys at a given time, the higher the tone you hear.

Now think about how this relates to our Flash sound object’s byte array.  We could actually draw a sine wave with numbers.  This is nothing new to programmers.  Folks use trigonometry all the time.  But, I personally, never thought to use it for sound.

Try this AS3 code:

for (var i:int=0; i < 100; i++ ) {
var sample:Number;
sample = Math.sin(i * 2 * Math.PI) * 50;
}

This will basically plot a picture like the one above – a sinewave.

If we made this into a sound, it almost, but not quite be a tone.  However, if we pop over to google, we can actually look up the frequency, of say…..a middle C note, or a middle A.

http://www.phy.mtu.edu/~suits/notefreqs.html

There’s one more piece to this puzzle though, and that’s sampling rate.  A digital audio file could have all the sinewaves, peaks and valleys in the world, but if your audio playback is super slow, it’s gonna sound like garbage.  That’s why we tell our audio software to read our sound at a certain speed.

If you’ve heard the term 44.1 kbps when folks talk about sample rate – you can look back at our little for loop that drew the sine wave, and realize that you need more than the 100 points of data as we drew, you need 44.1k, or 44,100 PER SECOND.

So let’s rewrite that code:

for (var i:int=0; i < durationinseconds * 44100; i++ ) {
var sample:Number;
sample = Math.sin((i) * 2*Math.PI/44100 * 440) * volume;
}

OK!  So now, we’re creating a one second middle A (4th octave).  Our sample rate in Flash is always going to be 44100 if coming from a flash sound object (though that’s not true for any audio file).  We learned from our handy/note frequency chart when looking on google that 440hz is a middle A.

As a side note, lets say we want a different octave.  To go lower, half the frequency to get 220.  To go lower, half that.  Higher?  Double it.

Let’s think about the voice now.  The nice sine wave will give a nice, round tone.  For a dirtier tone that sounds like it has sharp edges….well our sine wave needs to have sharp edges – which is actually a square wave.  Our nice round peaks and valleys would be just straight corners.

To change the tone in the code, try this:

sample = Math.sin((i) * 44100 * 2 * Math.PI * frequency) > 0 ? volume : -volume;

As for more voices, well, I haven’t tried it, but a real world instrument would have a hard strike and then some falloff.  So our nice sine wave would be less round at the start of each peak, but comes back down way slower.

Here’s my final code (and keep in mind that I’m writing the sample 2 times, one for the left channel and one for the right):

returnBytes:ByteArray = new ByteArray();
for (var i:int=0; i < _duration * 44100; i++ ) {
var sample:Number;
if (_voice==VOICE_SQUARE) {
sample = Math.sin((i) * 44100 * 2 * Math.PI * this.frequency) > 0 ? _amplitude : -_amplitude;
} else {
sample = Math.sin((i) * 44100 * 2 * Math.PI * this.frequency) * _amplitude;
}
returnBytes.writeFloat(sample);
returnBytes.writeFloat(sample);
}

Now, what do we do with that byte array?  Well, as of Flash 10, our sound object, has a sample data event.  When this event is called, when it needs new bytes, it’ll call out to your custom sample data method.

What I do – and what you may choose to do, or not choose to do, is make my whole byte array first, and then read sequential amounts of data into the byte array each time it’s called:

soundBytes.position = 0;
dynamicSound = new Sound();
dynamicSound.addEventListener(SampleDataEvent.SAMPLE_DATA, addSoundBytesToSound, false, 0, true);
soundchannel = dynamicSound.play();

private function addSoundBytesToSound(event:SampleDataEvent):void
{
var bytes:ByteArray = new ByteArray();
soundBytes.readBytes(bytes, 0, Math.min(soundBytes.bytesAvailable, 8 * 8192));
event.data.writeBytes(bytes, 0, bytes.length);
}

To explain, about the 8 * 8192….

8192 is the maximum amount of samples you can use for each sample data event.  However….each sample is a left and a right 4 byte float.  So that’s 8…..time 8192.

There’s tons of cool stuff you can do with this.  If you don’t believe me, look up Andre Michelle – he’s THE MAN when it comes to this stuff.

June 27, 2009

Working with the Flash Sound Object as a ByteArray

Filed under: flash/flex — admin @ 4:24 pm

Last year I was experimenting quite a bit with Flash’s computeSpectrum functionality.   With computeSpectrum, I could take a snapshot of a playing sound and get volumes for various frequencies of the snapshot so I can visualize the sound.

Now, I wanted to take it a step further and visualize the sound before it’s even played.  Of course doing this with computeSpectrum is impossible because the sound has to be played first.

Luckily, in Flash Player 10, Adobe introduced sound.extract();  This can take the sound object and turn it into a byteArray.  I came across an example by Thibault Imbert and of course the example worked very well to visualize a sound spectrum from a byte array – though I still wasn’t sure what each byte represented in the Flash Sound object.  And even though, the spectrum looked OK,  a comment from Schell explained that this is a false waveform and to look at his blog post.

The blog post provides some great working code – but there was quite a bit I didn’t understand yet.

So, I attacked the problem from another angle today.  I thought – OK, I don’t really understand the bytes that are coming OUT of the sound object, but maybe if I play with shoving bytes INTO a sound object to create dynamic sounds, I can get a better handle on tings.

The real struggle came because I used Christopher Martin-Sperry’s code to create a sound object from the raw byte array provided by an MP3 file.  I used this before really looking into how all this stuff works.  The code works fantastically – but put my brain on the wrong track.

I was on the wrong track because MP3′s and Flash’s sound Object work a wee bit differently from each other.  In an MP3 file, there is a header to give some info about the file.  Also, the MP3 file is broken into “frames”.  Each frame has a header to give information about the frame (bitrate and other things).

This doesn’t map so well to Flash’s sound object.  First of all, the bitrate of a Flash sound object looks like it doesn’t vary.  It seems to be 44.1 kilobytes per second all the time.  And it’s always in stereo.  It looks like the source of the audio will always get upconverted or downconverted to this 44.1KBps stereo format.

Not only that, but there’s no concept of frames.  I thought I’d need to know how to read a frame header, or at least know how long each frame is to know what type of data I’m getting.

But that doesn’t appear to be the case.  The byteArray extracted from a sound object appears to be an alternating left/right channel stream.

How does the stream relate to time? I felt a little dumb when I figured this one out – because it’s a little obvious.  It generated a few tones before having this little gem of an epiphany.

Well, if the Flash sound object is 44.1KBps, that’s 44,100 bytes per second.  So when reading the stream, we read 44,100 bytes before a second is up. Right?

Well, not so right it would seem.  You have to consider both channels.  You’d read each channel 44,100 times to make up a second, making one second 88,200….something.

What is that something?  Well my computer sciece teaching has failed me, but I can speculate.  Adobe provides a morse code generation sample on their developer network.

In it, there is a comment when they are writing the samples to the sound stream that they are writing 8192 samples to the stream.  However, the length of the byte array they are actually writing is 8192 x 8, explaining that each sample is two 4-byte floating point numbers.

So perhaps when reading floats with ByteArray.readFloat, you’re actually reading both left and right floats from the same byte.  If I’m right, then you’re still using 44,100 bytes per second, but each byte represents two floating point numbers.

However, it works out, I’ve noticed that when creating six 1 second tones, by writing two floats at a time, 44,100 times,  it clocks in at 6 seconds.  So infer whatever you wish from that – I know I have.

My next step is to figure out how to pull frequency data at a particular point in time.

Older Posts »

Powered by WordPress