Using WMP and Quicktime in Silverlight (Revisited)

November 6, 2011 at 10:34 am | Posted in Silverlight | 1 Comment

In 2007, I was working on a Silverlight application in which I wanted to change the rate of speech playback without affecting the pitch.  At that time, the media element available to Silverlight did not have a rate control while Windows Media Player works wonderfully for mp3 files.  I found that I could actually use WMP in a Silverlight application by embedding the player in the HTML file that hosted the Silverlight control and then controlling the player by using a bridge between managed C# and JavaScript.  The technique was documented in this blog post.

Since then, changes to Silverlight and browsers that have broken this code.

I resurrected the old code base and ported it to Silverlight 4.  The code can be tested here.  And you can download a zip file of the web project here.  The code is still a bit dated and ‘hackish’ but has been tested on IE9, Chrome 15 and Firefox 3.6 with the WMP plug-in installed.  For some reason, the volume control does not work in Chrome.  I also forced Chrome to use the Quicktime player instead of WMP.  It works, but Quicktime does not preserve the pitch when the rate is changed so it will sound weird at slow speeds.  I did not test on a Mac.

Silverlight 5 is supposed to have a new media element that has a rate control function.  If it works as well as WMP, then this approach will no longer be necessary.

Data binding made simple

September 1, 2011 at 9:51 pm | Posted in Silverlight, Uncategorized, WPF | Leave a comment

Data binding is a very important topic for WPF and Silverlight applications and Microsoft has created a very rich set of functionality in the frameworks to make it work in a variety of ways.  The problem is there are so many ways of doing things that I find the documentation from different sources to be quite confusing.

Today, I was working on a little application which should be very amenable to the use of data binding UI controls (listboxes and textboxes) to data objects.  After being away from WPF for a time, I was struggling to relearn exactly the best way to do it.  The MSDN documentation is thorough but is TMI at times.

I finally ran across a tutorial that captures the essence of what I needed to do in this application as well as others that I have worked on in the past.  If you want a clear example of how to implement data binding for either Silverlight or WPF, I recommend you check out this post. 

Synchronizing media with animation in Silverlight

November 4, 2010 at 9:22 am | Posted in Silverlight | 2 Comments

I am working on a project which consists of several rather long animations that I am building in Blend.  I ran into a problem in adding sound clips that should be played in specific points in the timeline.  Since storyboard only has one event (the end), there was no clear way of starting MediaElement at any specific point in a timeline.

I was directed to a couple of sites for solutions to this problem:

Solution 1

Solution 2

But I came up with a variation that fits my application a little better.

1. Add a textbox to the artboard with an opacity of zero.

2. Add a TextChanged handler for this textbox.

3. In the handler set up ‘if else’ constructs that launch media playback if a specific string is detected in the Text property.

4. In the timeline add keyframes on the textbox that change the Text property to your trigger strings at any point in the animation.

5. For the MediaElement, I set the AutoPlay to true.  Then whenever I change the media source property (in the TextChanged handler), the sound is played.

With this technique, a storyboard is started with various animations.  One of the items animated is the invisible textbox Text field.  When the animation changes the text to a specific keyword, the event handler for the textbox fires and the text field is compared with a list of keywords that trigger the loading of a corresponding media source to the MediaElement which then plays it.

Find the color of a point in a LinearGradientBrush

January 28, 2008 at 7:54 am | Posted in .Net, C#, Silverlight, WPF | 6 Comments

I became intrigued by the problem of finding the color at any point of a rectangle that has been filled with a LinearGradientBrush.  There is are techniques for doing this by essentially sampling the pixel at a rendered point but I wanted to approach the problem algorithmically.

I developed the following method which receives a rectangle object that has been filled with a LinearGradientBrush and a point relative to the upper left corner of the rectangle and returns the calculated color at that point.  There are no restrictions on the Start/Stop Points or the number of GradientStops.  The ColorInterpolationMode can be either SRgbLinearInterpolation (default) or ScRgbLinearInterpolation.  The GradientSpreadMethod is required to be the default (Pad).

There are two methods below:

Color GetColorAtPoint(Rectangle r, Point p) – finds the color at any point of a LinearGradientBrush filled rectangle.

private Double dist(Point px, Point po, Point pf) – helper method for GetColorAtPoint.

To use, copy both methods to your class and call GetColorAtPoint() with a rectangle and point.

//Calculates the color of a point in a rectangle that is filled
//with a LinearGradientBrush.
private Color GetColorAtPoint(Rectangle theRec, Point thePoint)
{
    //Get properties
    LinearGradientBrush br = (LinearGradientBrush)theRec.Fill;
 
    double y3 = thePoint.Y;
    double x3 = thePoint.X;
 
    double x1 = br.StartPoint.X * theRec.ActualWidth;
    double y1 = br.StartPoint.Y * theRec.ActualHeight;
    Point p1 = new Point(x1, y1); //Starting point
 
    double x2 = br.EndPoint.X * theRec.ActualWidth;
    double y2 = br.EndPoint.Y * theRec.ActualHeight;
    Point p2 = new Point(x2, y2);  //End point
 
    //Calculate intersecting points 
    Point p4 = new Point(); //with tangent
 
    if (y1 == y2) //Horizontal case
    {
        p4 = new Point(x3, y1);
    }
 
    else if (x1 == x2) //Vertical case
    {
        p4 = new Point(x1, y3);
    }
 
    else //Diagnonal case
    {
        double m = (y2 - y1) / (x2 - x1);
        double m2 = -1 / m;
        double b = y1 - m * x1;
        double c = y3 - m2 * x3;
 
        double x4 = (c - b) / (m - m2);
        double y4 = m * x4 + b;
        p4 = new Point(x4, y4);
    }
 
    //Calculate distances relative to the vector start
    double d4 = dist(p4, p1, p2);
    double d2 = dist(p2, p1, p2);
 
    double x = d4 / d2;
 
    //Clip the input if before or after the max/min offset values
    double max = br.GradientStops.Max(n => n.Offset);
    if (x > max)
    {
        x = max;
    }
    double min = br.GradientStops.Min(n => n.Offset);
    if (x < min)
    {
        x = min;
    }
 
    //Find gradient stops that surround the input value
    GradientStop gs0 = br.GradientStops.Where(n => n.Offset <= x).OrderBy(n => n.Offset).Last();
    GradientStop gs1 = br.GradientStops.Where(n => n.Offset >= x).OrderBy(n => n.Offset).First();
 
    float y = 0f;
    if (gs0.Offset != gs1.Offset)
    {
        y = (float)((x - gs0.Offset) / (gs1.Offset - gs0.Offset));
    }
 
    //Interpolate color channels
    Color cx = new Color();
    if (br.ColorInterpolationMode == ColorInterpolationMode.ScRgbLinearInterpolation)
    {
        float aVal = (gs1.Color.ScA - gs0.Color.ScA) * y + gs0.Color.ScA;
        float rVal = (gs1.Color.ScR - gs0.Color.ScR) * y + gs0.Color.ScR;
        float gVal = (gs1.Color.ScG - gs0.Color.ScG) * y + gs0.Color.ScG;
        float bVal = (gs1.Color.ScB - gs0.Color.ScB) * y + gs0.Color.ScB;
        cx = Color.FromScRgb(aVal, rVal, gVal, bVal);
    }
    else
    {
        byte aVal = (byte)((gs1.Color.A - gs0.Color.A) * y + gs0.Color.A);
        byte rVal = (byte)((gs1.Color.R - gs0.Color.R) * y + gs0.Color.R);
        byte gVal = (byte)((gs1.Color.G - gs0.Color.G) * y + gs0.Color.G);
        byte bVal = (byte)((gs1.Color.B - gs0.Color.B) * y + gs0.Color.B);
        cx = Color.FromArgb(aVal, rVal, gVal, bVal);
    }
    return cx;
}
 
//Helper method for GetColorAtPoint
//Returns the signed magnitude of a point on a vector with origin po and pointing to pf
private double dist(Point px, Point po, Point pf)
{
    double d = Math.Sqrt((px.Y - po.Y) * (px.Y - po.Y) + (px.X - po.X) * (px.X - po.X));
    if (((px.Y < po.Y) && (pf.Y > po.Y)) ||
        ((px.Y > po.Y) && (pf.Y < po.Y)) ||
        ((px.Y == po.Y) && (px.X < po.X) && (pf.X > po.X)) ||
        ((px.Y == po.Y) && (px.X > po.X) && (pf.X < po.X)))
    {
        d = -d;
    }
    return d;
}

The method works by projecting the input point onto the vector that is described by the LinearGradientBrush Start/Stop points.  There are special cases for horizontal and vertical lines.  The width of the line is calculated as well as the relative distance of the projected point from the StartPoint (using the dist() method).  The GradientStops are ordered and two stops are found that are the closest the projected point.  The color channel values are interpolated based on the relative position of the projected point and these two stops.  The values are assembled into a color and returned.

You can download a demo of this method here.

Kaxaml: The XAML editor for me

January 11, 2008 at 8:38 am | Posted in Silverlight, WPF | Leave a comment

I highly recommend Kaxaml for your XAML editor.  Not only does it look and work great but it also has some very useful features not found on other XAML editors.

I won’t bother listing all of the features that you can read about here, but my favorites are Intellisense support (with code completion!), snippets including a generous collection of simple control styles and the ability to create and save your own, drag and drop, a beautiful color picker, and a XAML scrubber.

With other XAML editors, I thought, “what’s the point? Just use the designer in Visual Studio.”  But Kaxaml is one tool that I will use first to build and customize my UIs–largely due to its snippets, colorpicker and ease of use.  (I don’t use Blend.)

Download Kaxaml here: www.kaxaml.com

Using WMP and Quicktime with Silverlight

August 15, 2007 at 7:47 am | Posted in Silverlight | 9 Comments

 

Note: There have been some breaking changes in Silverlight since this post was published.  For the latest info see this link.

Silverlight has a MediaElement control  which can be used for most cross-platform media display tasks.  I was disappointed that my application can not use it because it currently does not support playback at a variable rate.

My application requires a media player that can playback speech mp3 files at rates varying from 0.5 to 1.0x realtime.  Windows Media Player (WMP) is especially good at this in that the playback at reduced speed does not modify the pitch of the speech.

My solution to this problem is to instantiate a WMP or Quicktime player in the page depending on the platform and control the player from Silverlight C# code through a Javascript layer.

First, the player is installed in the body of the html page like this:

<!--Player goes here-->
     <div id="PlayerDiv"></div>    

 <script type="text/javascript">  

 //Installs the player
 var wmp;
 var qtp;
 document.getElementById('PlayerDiv').innerHTML = playerCreate('news clip.mp3');
 if(useWmp){
 wmp = document.getElementById('wmPlayer');
 }else{
 qtp = document.qtPlayer;
 }    

 </script>

The playerCreate function is a bit of Javascript that determines the browser capabilities and installs either a WMP or Quicktime player.  If IE is detected or Firefox is detected with the WMP component, then WMP is installed else use Quicktime.  (WMP is preferred over Quicktime for this application because while Quicktime has a rate control, the pitch is not preserved for mp3 playback.)  Here is the playerCreate Javascript:

var useWmp;
    function playerCreate(url) {
    //Find browser
var userAgent = navigator.userAgent.toLowerCase();
var is_opera  = (userAgent.indexOf('opera') != -1);
var is_saf    = ((userAgent.indexOf('applewebkit') != -1) || (navigator.vendor == "Apple Computer, Inc."));
var is_webtv  = (userAgent.indexOf('webtv') != -1);
var is_ie     = ((userAgent.indexOf('msie') != -1) && (!is_opera) && (!is_saf) && (!is_webtv));
var is_ie4    = ((is_ie) && (userAgent.indexOf("msie 4.") != -1));
var is_moz    = ((navigator.product == 'Gecko') && (!is_saf));
var is_kon    = (userAgent.indexOf('konqueror') != -1);
var is_ns     = ((userAgent.indexOf('compatible') == -1) && (userAgent.indexOf('mozilla') != -1) && (!is_opera) && (!is_webtv) && (!is_saf));
var is_ns4    = ((is_ns) && (parseInt(navigator.appVersion) == 4));   

    var str = "";
    if (is_ie) {
    // create the WMP for IE 
    useWmp = true;
    str = '<object id="wmPlayer" classid="CLSID:6BF52A52-394A-11d3-B153-00C04F79FAA6" width="0" height="0">';
// str += '<param name="URL" value="'+url+'" />'; 
    str += '<param name="uiMode" value="none">';
    str += '<param name="autoStart" value="false">';
    str += '</object>';
    return str;  

    }else if(is_moz && navigator.plugins["Microsoft® Windows Media Player Firefox Plugin"]){
    // create WMP for FF. 
    useWmp = true;
    str = '<object id="wmPlayer" type="application/x-ms-wmp" data="'+url+'" width="0" height="0">';
// str += '<param name="URL" value="'+url+'" />'; 
    str += '<param name="uiMode" value="none">';
    str += '<param name="autoStart" value="false">';
    str += '</object>';
    return str;      

    }else{
    //try to create the qtp as last resort
    useWmp = false;
    str ='<object id="qtPlayer" classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B" codebase="http://www.apple.com/qtactivex/qtplugin.cab" width="0" height="0" >';
// str += '<param name="src" value="'+url+'">';
    str += '<param name="autoplay" value="false">';
    str += 'http://qtStartUp.mp3';
    str += ' </embed> ';
    str += '</object> ';
    return str;  

}
}

A set of Javascript functions were defined in order to create a common API for the Silverlight C# code to use.

function playerPlay(){
if(useWmp){
wmp.controls.play();
}else{
qtp.Play();
}
}  

function playerStop(){
if(useWmp){
wmp.controls.stop();
}else{
qtp.Stop();
qtp.Rewind();
}
}   

function playerPause(){
if(useWmp){
wmp.controls.pause();
}else{
qtp.Stop();
}
}   

function playerUrl(url){  

if(useWmp){  

wmp.URL = url;
}else{
qtp.SetURL(url);
qtp.Stop();
}
}   

function playerGetPosition(){
if(useWmp){
return wmp.controls.currentPosition;
}else{
return qtp.GetTime()/600;
}
}   

function playerSetPosition(t){
if(useWmp){
wmp.controls.currentPosition = t;
}else{
qtp.SetTime(t*600); //int
}
}   

function playerSetVolume(v){
if(useWmp){
wmp.settings.volume = v;
}else{
qtp.SetVolume(v); //int 0-100
}
}   

function playerSetRate(r){
if(useWmp){
wmp.settings.rate = r;
}else{
qtp.SetRate(r); //flost 0.5-1.0
}
}

These functions are called by a single event handler that manages calls from the Silverlight C# code using the techniques for C# to Javascript communications described in the QuickStart guide.

//Event handler for Silverlight control
function agEventHandler(sender, args){
switch (args.Name){
case 'stop':
playerStop();
break;
case 'play':
playerPlay();
break;
case 'pause':
playerPause();
break;
case 'url':
playerUrl(args.Sdata);
break;
case 'position':
playerSetPosition(args.Data);
break;
case 'volume':
playerSetVolume(args.Data);
break;
case 'rate':
playerSetRate(args.Data);
break;  

default:
}

The event handler arguments are defined on the C# side as follows:

//This class defines the arguments for the control's events
[Scriptable]
public class CustomEventArgs : EventArgs
{
    private double data;
    private string name;
    private string sdata;  

    public CustomEventArgs(string iname, double idata, string isdata)
    {
        data = idata;
        name = iname;
        sdata = isdata;
    }  

    [Scriptable]
    public string Name
    {
        get { return name; }
        set { name = value; }
    }  

    [Scriptable]
    public double Data
    {
        get { return data; }
        set { data = value; }
    }  

    [Scriptable]
    public string Sdata
    {
        get { return sdata; }
        set { sdata = value; }
    }
}//End of class

This class of custom event arguments allows the passing of a string name which specifies the media player event to control as well as a double and string parameter for data.

The event method is defined in the C# code like this:

//This defines the event from the control
//This is an example of how to fire an event from the control to Javascript
//AgEvents(this, new CustomEventArgs("hello", 25, "something"));
[Scriptable]
public event EventHandler<CustomEventArgs> AgEvents;

Finally, here is an example of C# code controlling the media player:

void play()
{
    if (playerState != (int)playerStates.paused)
    {
        AgEvents(this, new CustomEventArgs("position", t1, null));
    }  

    AgEvents(this, new CustomEventArgs("play", 0, null));

 

Hopefully, future releases of Silverlight will have a MediaElement with a rate control so that my application can be greatly simplified.

Next Page »

Create a free website or blog at WordPress.com.
Entries and comments feeds.