Blog

Charlie

ConnectyCube full backend update

Posted by Charlie on May 23, 2019

Hello!

We are more than happy to announce full backend update of ConnectyCube server, which includes full code re-write and optimization along with admin panel re-branding.

We decided to leave connection algorithm as it is for your convenience, so no worries – no need to change anything in your application for now. However all internal algorithms were changed, including some chat functionality, calling functionality and backend infrastructure. If you’re considering Enterprise plan please contact us to get new requirements for your hosting providers and feel free to ask any questions about how our new API works.

With proud we can state that speed has increased by 30% approximately, which means faster chats, faster response time and general performance improvement.

Another good point is that we have improved stats collecting, so that you can always have up-to-date information about your application performance, your account usage and so on. It is now available in ‘Account Stats’ menu in your admin panel. You can easily switch between applications to check stats for each of them.

 

The link to admin panel remains the same https://admin.connectycube.com Feel free to explore all new functionality right away.

We are grateful to all our customers, who sent us the feedback during these months and took part in beta-testing. Without you we could not improve our services and achieve what we have now.

Feel free to contact us in case of any questions or suggestions, we’re always here to help!

iris

New Android SDK 1.8.0

Posted by iris on April 16, 2019

Hi,

We are glad to let you know that ConnectyCube Android SDK 1.8.0 has beed released!
Here is the change log:

Chat module

Updated

XMPP Smack lib upgrade to 4.3.2

In ConnectycubeRosterEntry:

removed getStatus(), now use isSubscriptionPending() instead.

Fixed

In ConnectycubeRestChatService:

parsing Successfully_deleted ids field when deleteDialogs()

New SDK is available on our Github page.

Stay tuned!

Charlie

ReactNative video chat code sample

Posted by Charlie on April 8, 2019

Hi,

We are happy to introduce our ReactNative video chat code sample, which you can use with your applications as basis or example of correct logic.

Basic UI looks the following way:

Code sample and basic instructions are available on our GitHub page.

Feel free to check ConnectyCube React Native getting started and Video Chat API documentation to ease the development.

Stay tuned! More features are coming!

iris

How to Add Snapchat-like Video Filters with ConnectyCube

Posted by iris on April 2, 2019

Privacy becomes very popular nowadays.
People want to protect their private life and stay anonymous in many cases.
In some chats people would like to stay incognito or apply any other filters to their faces just for fun.

ConnectyCube SDK provides an interface to access video frames so then we can apply OpenCV filters/mask on it and have something like this in 1-1 chat:

Putting mask on face

Wondering how it can be implemented?

Read on!

Android

The following things need to be done on Android platform

1. Connect OpenCV SDK
Import OpenCV Module into your Android project according to A Beginner’s Guide to Setting up OpenCV Android Library on Android Studio

 

2. In ConnectyCube SDK you need to implement a callback on recipient’s side to access video frames, so then we can apply any mask on it:

        videoTrack.addRenderer(new VideoSink() {
            @Override
            public void onFrame(VideoFrame videoFrame) {
                     // apply OpenCV code here 
        // …

                     // render result on a view
      // RTCSurfaceView.onFrame(videoFrame);
            }
        });


More info on ConnectyCube VideoChat API is here

3. Detect face and apply a mask on it with openCV as follows:

Mat putMask(Mat src, Point center, Size face_size){

//mask : masque chargé depuis l'image
Mat mask_resized = new Mat(); //masque resizé
src_roi = new Mat(); //ROI du visage croppé depuis la preview
roi_gray = new Mat();


Imgproc.resize(mask ,mask_resized,face_size);

// ROI selection
roi = new Rect((int) (center.x - face_size.width/2), (int) (center.y - face_size.height/2),(int) face_size.width, (int) face_size.height);
//Rect roi = new Rect(10, 10, (int) face_size.width, (int) face_size.height);

src.submat(roi).copyTo(src_roi);

Log.e(TAG, "MASK SRC1 :"+ src_roi.size());

// to make the white region transparent
Mat mask_grey = new Mat(); //greymask
roi_rgb = new Mat();
Imgproc.cvtColor(mask_resized,mask_grey, Imgproc.COLOR_BGRA2GRAY);
Imgproc.threshold(mask_grey,mask_grey,230,255, Imgproc.THRESH_BINARY_INV);

ArrayList maskChannels = new ArrayList<>(4);
ArrayList result_mask = new ArrayList<>(4);
result_mask.add(new Mat());
result_mask.add(new Mat());
result_mask.add(new Mat());
result_mask.add(new Mat());

Core.split(mask_resized, maskChannels);

Core.bitwise_and(maskChannels.get(0),mask_grey, result_mask.get(0));
Core.bitwise_and(maskChannels.get(1),mask_grey, result_mask.get(1));
Core.bitwise_and(maskChannels.get(2),mask_grey, result_mask.get(2));
Core.bitwise_and(maskChannels.get(3),mask_grey, result_mask.get(3));

Core.merge(result_mask, roi_gray);        

Core.bitwise_not(mask_grey,mask_grey);

ArrayList srcChannels = new ArrayList<>(4);
Core.split(src_roi, srcChannels);
Core.bitwise_and(srcChannels.get(0),mask_grey, result_mask.get(0));
Core.bitwise_and(srcChannels.get(1),mask_grey, result_mask.get(1));
Core.bitwise_and(srcChannels.get(2),mask_grey, result_mask.get(2));
Core.bitwise_and(srcChannels.get(3),mask_grey, result_mask.get(3));

Core.merge(result_mask, roi_rgb);          

Core.addWeighted(roi_gray,1, roi_rgb,1,0, roi_rgb); 

roi_rgb.copyTo(new Mat(src,roi));

return src;}


Read more here.

Note:

All the above is implemented on a recipient’s side, so a sender has to notify the recipient that he/she enabled a mask on their side. You can set such notification via ConnectyCube System Messages API.

 

iOS

The same flow is applied to iOS – ConnectyCube iOS SDK also provides an access to video frames via CMSampleBufferRef.

1. Connect OpenCV SDK
Import OpenCV Module into your iOS project according to this guide.

2. Convert CMSampleBufferRef to cv::Mat: according to the suggestions here.  

- (cv::Mat)matFromBuffer:(CMSampleBufferRef)buffer {
    CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(buffer);
    CVPixelBufferLockBaseAddress( pixelBuffer, 0 );

    //Processing here
    int bufferWidth = (int)CVPixelBufferGetWidth(pixelBuffer);
    int bufferHeight = (int)CVPixelBufferGetHeight(pixelBuffer);
    unsigned char *pixel = (unsigned char *)CVPixelBufferGetBaseAddress(pixelBuffer);

    //put buffer in open cv, no memory copied
    cv::Mat mat = cv::Mat(bufferHeight,bufferWidth,CV_8UC4,pixel,CVPixelBufferGetBytesPerRow(pixelBuffer));

    //End processing
    CVPixelBufferUnlockBaseAddress( pixelBuffer, 0 );

    cv::Mat matGray;
    cvtColor(mat, matGray, CV_BGR2GRAY);

    return matGray;
}

 

3. Detect face with openCV according to this guide.

4. Apply a mask on face with OpenCV according to suggestions here.

 

Note:

All the above is implemented on a recipient’s side, so a sender has to notify the recipient that he/she enabled a mask on their side. You can set such notification via ConnectyCube System Messages API.

 

Good luck!

Have any questions or need help?

Feel free to contact us.