Oct 7, 2022
Building a Twitter Spaces clone in Flutter
A walkthrough of how we built SpeakUp, a clone of Twitter Spaces we created for conducting spaces of our internal communities.
Ashish Rajesh GourSoftware Engineer - II
Introduction
Audio streaming has garnered a lot of attention recently. You’ve heard of audio streaming platforms such as Clubhouse and Twitter Spaces. As part of our flutter community initiatives at Geekyants, we started experimenting with 100ms.live and built a clone around Twitter Spaces.
Twitter Spaces is a new feature that allows users to have live audio conversations on the platform. Users can host these conversations in an audio chat room called a "Space" and invite other users to participate.
This article will demonstrate how to build a Twitter Spaces clone with the 100ms Flutter SDK.
Why 100ms?
100ms is a cloud-based live video infrastructure software that allows developers to create video and audio conferencing applications on the web, iOS, and Android platforms using the 100ms software development kit (SDK), REST APIs, and account dashboard. The platform provides an easy and flexible kit to enable developers to create awesome, feature-rich, and custom-built video and audio conferencing applications without breaking much sweat.
Prerequisites
To follow this tutorial, you must have a basic understanding of
Basic Terms to Know in the 100 ms SDK
Basic Terms to Know in the 100 ms SDK
Before starting, let’s get acquainted with some common terms that will be frequently used throughout this article :
Setting up the project
Create A Template : To get started on this project, create an account on the 100ms dashboard. Moving forward, we would create a template where we will define the roles and settings, simply select “Video Conferencing”. The roles to be defined for this project are the host and guest. Each role has its permissions and settings with the host having the highest set of permissions.
Create A Room : Now, we have to create a room based on the template created above, as earlier mentioned, a room is the object returned when a connection to the SDK is successful, it defines the place where you and other participants meet and interact in the app. To create a room, go to the Rooms tab and click on “Create A Room”, and input the room name. On successful creation of a room, a room id is automatically generated which will be useful while rendering the rooms on UI.
Integrate the SDK
To get started, we’ll need to add the Flutter package for 100ms SDK. You can find it on pub.dev or run the following command in the terminal:
flutter pub add hmssdk_flutter
We’ll also need to add a few other packages to pubspec.yaml file namely:
hmssdk_flutter: ^0.7.3
flutter_bloc: ^8.0.1
firebase_core: ^1.19.1
firebase_auth: ^3.4.1
google_sign_in: ^5.3.3
http: ^0.13.4
cloud_firestore: ^3.2.1
flutter_secure_storage: ^5.0.2
permission_handler: ^10.0.0
Setting up services
Create a new folder services with a file sdk_initializer.dart in it.
Add the following code to initialize the SDK:
import 'package:hmssdk_flutter/hmssdk_flutter.dart';
class SdkInitializer {
static HMSSDK hmssdk = HMSSDK();
}
Add Permissions
As discussed in the above terms a track represents either the audio or video that a peer is publishing.
You will require Recording Audio and Internet permission in this project as we are focused on the audio track in this tutorial.
Android Permissions
Add the following permissions in your AndroidManifest file (android/app/src/main/AndroidManifest.xml):
<uses-feature android:name="android.hardware.camera"/>
<uses-permission android:name="android.permission.CAMERA"/>
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE"/>
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/>
<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
iOS Permissions
Add the following permissions to your Info.plist file:
<key>NSCameraUsageDescription</key>
<string>Allow access to camera to capture images for profile.</string>
<key>NSLocalNetworkUsageDescription</key>
<string>Allow access to the internet to enable audio calling.</string>
<key>NSMicrophoneUsageDescription</key>
<string>Allow access to mic to enable audio calling.</string>
Getting the permissions from users
To use the camera or microphone in our app, we need to get the required permissions from the users.
You can use the permission_handler package to do that easily.
flutter pub add permission_handler
Call the getPermissions() function inside the initState() of your Stateful HomeScreen.
void getPermissions() async {
await Permission.camera.request();
await Permission.microphone.request();
await Permission.bluetooth.request();
while ((await Permission.camera.isDenied)) {
await Permission.camera.request();
}
while ((await Permission.bluetooth.isDenied)) {
await Permission.camera.request();
}
while ((await Permission.microphone.isDenied)) {
await Permission.microphone.request();
}
}
@override
void onJoin({required HMSRoom room}) {
bloc.add(OnJoinRoomEvent(room: room));
}
Implement Listener
Now, we have to implement a listener class over the current SDK which will help us to interact with the SDK easily.
HMSUpdateListener listens to all the updates happening inside the room. 100ms SDK provides callbacks to the client app about any change or update happening in the room after a user( peer) has joined by implementing HMSUpdateListener.
class HMSInteractor implements HMSUpdateListener {
HMSInteractor({required this.bloc});
final RoomBloc bloc;
When someone requests a track change for video or audio this will be triggered. Where [hmsTrackChangeRequest] request instance consists of all the required info about track changes.
@override
void onChangeTrackStateRequest({required HMSTrackChangeRequest hmsTrackChangeRequest}) {}
This will be called on a successful joining of the room by the user.
@override
void onMessage({required HMSMessage message}) {
bloc.add(OnHMSMessageEvent(message: message));
}
This is called when there is a new broadcast message from any other peer in the room. This can be used to implement chat in the room.
@override
void onReconnected() {
print("onReconnected");
}
This will be called whenever there is an update on an existing peer, a new peer got added or an existing peer is removed. This callback can be used to track all the peers in the room.
@override
void onPeerUpdate({required HMSPeer peer, required HMSPeerUpdate update}) {
if ((update == HMSPeerUpdate.peerJoined) || (update == HMSPeerUpdate.peerLeft))
bloc.add(OnPeerUpdateEvent(peer, update));
}
This method is called when a peer is back in the room after reconnection.
@override
void onRemovedFromRoom({required HMSPeerRemovedFromPeer hmsPeerRemovedFromPeer}) {
if (hmsPeerRemovedFromPeer.roomWasEnded) {
bloc.add(EndRoomByRemoteHostEvent());
}
}
This method is called when a network or some other error happens.
@override
void onReconnecting() {
print("onReconnecting");
}
This method is called when the host removes you or when someone ends the room at that time, it gets triggered.
@override
void onRoomUpdate({required HMSRoom room, required HMSRoomUpdate update}) async {}
This method is called when someone requests a change of role.
@override
void onRoleChangeRequest({required HMSRoleChangeRequest roleChangeRequest}) {}
This is called when there is a change in any property of the Room.
This is called when there are updates on an existing track or a new track got added/an existing track is removed. This callback can be used to render the video on the screen whenever a track gets added.
@override
void onTrackUpdate({required HMSTrack track, required HMSTrackUpdate trackUpdate, required HMSPeer peer}) {
print("onTrackUpdate");
}
Note: An HMSSpeaker object contains -
@override
void onUpdateSpeakers({required List<HMSSpeaker> updateSpeakers}) {
bloc.add(OnUpdateSpeakersEvent(updateSpeakers));
}
This will be called when there is an error in the system and SDK has already retried to fix the error.
@override
void onHMSError({required HMSException error}) {
print(error.message);
}
@override
void onAudioDeviceChanged({HMSAudioDevice? currentAudioDevice, List<HMSAudioDevice>? availableAudioDevice}) {}
}
Joining the Room
Now, to join an ongoing space, we can call the onJoin() method on HMSSDK with the config settings. This would require an authentication token and a room id.
To get an auth token, we need to send an HTTP post request to the Token endpoint which can be obtained from the dashboard manually.
Go to Developer -> Copy Token endpoint (under Access Credentials)
Future<TokenResponse?>
getAppToken({required String roomId, required String user, String role = "host"}) async {
try {
final response =
await http.post(Uri.parse
("${RoomConstants.appTokenUrl}?roomId=$roomId&&user=$user&&role=$role"));
final jsonResponse = json.decode(response.body);
final TokenResponse tokenResponse = TokenResponse.fromJson(jsonResponse);
return tokenResponse;
} catch (e) {
return null;
}
}
A successful HTTP post would return us a token that can be passed to the join method of HMSSDK as config.
on<JoinRoomEvent>((event, emit) async {
emit(JoinRoomLoadingState());
if (event.room?.roomId != null && event.room?.roomId != room.roomId && room.isRoomJoined) {
room.isRoomJoined = false;
room.isHandRaised = false;
room.chats = [];
SdkInitializer.hmssdk.leave();
}
if (room.isRoomJoined) {
emit(JoinRoomState());
emit(UpdateRoomUI());
return;
}
RoomDetails? roomDetails = event.room;
if (roomDetails == null) {
roomDetails = room;
} else {
room = roomDetails;
}
final TokenResponse? tokenResponse = await roomRepository.getAppToken(
user: room.userId ?? '', roomId: room.roomId ?? '', role: event.role ?? 'host');
final username = FirebaseAuth.instance.currentUser?.displayName?.split(" ").join("");
final Map<String, dynamic> metadata = Metadata(photoURL: FirebaseAuth.instance.currentUser?.photoURL).toJson();
if (tokenResponse == null) {
emit(JoinRoomError("Failed to Join room"));
return;
}
HMSConfig hmsConfig = HMSConfig(
metaData: json.encode(metadata),
authToken: tokenResponse.token!,
userName: "$username",
);
await SdkInitializer.hmssdk.join(config: hmsConfig);
emit(JoinRoomState());
emit(OnJoinRoomLoadingState());
});
On success, JoinRoomState() will be fired by the bloc which will render the updated UI.
@override
Widget build(BuildContext context) {
return BlocConsumer<RoomBloc, RoomState>(
listener: (context, state) async {
if (state is EndRoomByRemoteHostEventState) {
GoRouter.of(context).go(bottomNavRoute);
}
},
buildWhen: (previous, current) =>
previous != current &&
(current is UpdateRoomUI || current is OnJoinRoomLoadingState || current is JoinRoomState),
builder: (context, state) {
room = BlocProvider.of<RoomBloc>(context).room;
if (state is JoinRoomLoadingState) {
return SafeArea(
child: Scaffold(
body: Center(
child: CircularProgressIndicator(
color: kWhite,
)),
),
);
}
bool isRoomJoined = room.isRoomJoined;
room.remotePeers?.sort((a, b) => a.role.name.length.compareTo(b.role.name.length));
return Scaffold(
appBar: buildAppBar(),
bottomNavigationBar: CustomBottomNavigationBar(showChat: showChat, toggleChat: toggleChat),
body: Padding(
padding: const EdgeInsets.fromLTRB(20, 10, 20, 20),
child: Column(children: [
buildRoomDetails(),
const SizedBox(height: 20),
if (showChat == false && isRoomJoined)
Expanded(
flex: 1,
child: GridView.count(
crossAxisCount: 4,
mainAxisSpacing: 5,
childAspectRatio: 1 / 1.1,
children: [
...List.generate((room.remotePeers ?? []).length, (index) {
return buildRoomParticipants((room.remotePeers ?? [])[index]);
}),
],
),
),
if (showChat && isRoomJoined)
Expanded(
flex: 1,
child: Chats(
chats: room.chats,
showChat: showChat,
toggleChat: toggleChat,
addChat: addChat,
)),
if (!isRoomJoined)
Center(
child: CircularProgressIndicator(
color: kWhite,
))
]),
),
);
},
);
}
✅ If successful, the function onJoin(room: HMSRoom) method of HMSUpdateListener will be invoked with details about the room containing in the HMSRoom object.
❌ If failure, the function onError(error: HMSException) method will be invoked with failure reason.
Creating a Room
For creating a room add the following lines of code in room_repository.dart file.
Future<CreateRoomResponse?> createRoom({required Room room}) async {
try {
// get management token from localStorage
const storage = FlutterSecureStorage();
final token = await storage.read(key: "management_token");
// create room
final response = await http.post(Uri.parse(
"${RoomConstants.createRoomUrl}?template=twitterspaces_videoconf_a69e75f3-6970-451f-bc3f-c5d95cc950ba&&token=$token&&name=${room.name}"));
final jsonResponse = json.decode(response.body);
final CreateRoomResponse createRoomRespone = CreateRoomResponse.fromJson(jsonResponse);
createRoomRespone.private = room.private;
await saveRoomDetails(createRoomResponse: createRoomRespone, room: room);
return createRoomRespone;
} catch (e) {
return null;
}
}
When an user presses the create room button, we will send the CreateRoomEvent to the RoomBloc to handle it and emit the RoomCreated State.
on<CreateRoomEvent>((event, emit) async {
emit(CreateRoomLoadingState());
final username = FirebaseAuth.instance.currentUser?.displayName?.split(" ").join("");
CreateRoomResponse? createRoomResponse = await roomRepository.createRoom(room: event.room);
if (createRoomResponse == null) {
emit(CreateRoomError("Failed to create room"));
return;
}
RoomDetails roomDetails = RoomDetails(
approved: [],
roomName: event.room.name,
username: username,
private: event.room.private,
members: [],
coverPhoto: event.room.coverPhoto,
roomId: createRoomResponse.room?.id,
userId: createRoomResponse.room?.user,
active: createRoomResponse.room?.active,
customerId: createRoomResponse.room?.customer,
createdAt: createRoomResponse.room?.createdAt,
chats: [],
remotePeers: [],
requests: [],
speakers: []);
room = roomDetails;
emit(RoomCreatedState());
});
✅ On success, RoomCreatedState() will get triggered and a new room card will be rendered on the home screen.
@override
Widget build(BuildContext context) {
return InkWell(
onTap: () {
BlocProvider.of<RoomBloc>(context).add(
JoinRoomEvent(
room: widget.room, role: FirebaseAuth.instance.currentUser?.uid == widget.room?.uid ? 'host' : 'guest'),
);
},
child: Container(
margin: EdgeInsets.all(16),
decoration: BoxDecoration(borderRadius: BorderRadius.circular(6), color: kGoLiveInactive),
child: Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: [
RoomCardImage(
url: widget.room?.coverPhoto,
),
Padding(
padding: EdgeInsets.all(15),
child: Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: [
Text(
widget.room?.roomName ?? '',
style: TextStyle(
overflow: TextOverflow.ellipsis,
fontWeight: FontWeight.bold,
fontSize: 16,
color: kHappeningNow),
),
SizedBox(
height: 8,
),
Row(
mainAxisAlignment: MainAxisAlignment.spaceBetween,
crossAxisAlignment: CrossAxisAlignment.center,
children: [
Expanded(
flex: 1,
child: Row(
crossAxisAlignment: CrossAxisAlignment.center,
children: [
CircleAvatar(
radius: 20,
backgroundImage: (user != null && user?.photoUrl != null)
? NetworkImage(user!.photoUrl!)
: NetworkImage(kHostImage),
backgroundColor: kTransparent,
),
SizedBox(
width: 6,
),
Text(
"@${widget.room?.username}",
overflow: TextOverflow.ellipsis,
style: TextStyle(color: kHappeningNow, fontWeight: FontWeight.w400, fontSize: 12),
)
],
),
),
Expanded(
flex: 1,
child: FacePile(
room: widget.room,
),
)
],
),
],
),
)
],
),
));
}
Render the Peers
A peer is an object returned by 100ms SDKs that hold the information about a user in room ( name, role, track, raise hand, message, etc.)
There a two types of peers available:
You can access the localPeer and remotePeers from hmssdk as shown in the code below:
final HMSLocalPeer? localPeer = await SdkInitializer.hmssdk.getLocalPeer();
final List<HMSPeer>? remotePeers= await SdkInitializer.hmssdk.getPeers();
Whenever a user (peer) rejoins, leaves or send message to another peer in a room OnJoinRoomEvent gets triggered which emits the UpdateRoomUI state which updates the UI.
on<OnJoinRoomEvent>((event, emit) async {
final hmsSDK = SdkInitializer.hmssdk;
HMSPeer? localPeer = await hmsSDK.getLocalPeer();
room.remotePeers = await hmsSDK.getPeers();
room.localPeer = localPeer;
try {
Map<String, dynamic> peerData = CustomHMSPeer(
peerId: localPeer?.peerId,
name: localPeer?.name,
isLocal: localPeer?.isLocal,
role: localPeer?.role.name,
customerUserId: localPeer?.customerUserId,
metadata: Metadata.fromJson(json.decode(localPeer?.metadata ?? "")))
.toJson();
roomsCollection = room.time != null
? await FirebaseFirestore.instance.collection('upcoming-rooms')
: await FirebaseFirestore.instance.collection('rooms');
final logsCollection = await FirebaseFirestore.instance.collection('logs');
if (localPeer?.role.name == 'host')
await roomsCollection?.doc(room.roomId).update({
'speakers': FieldValue.arrayUnion([localPeer!.peerId]),
});
await logsCollection.doc(room.roomId).update({
'logs': FieldValue.arrayUnion([
{'time': DateTime.now().toString(), 'peer': peerData}
])
});
hmsSDK.switchVideo(isOn: true);
hmsSDK.switchAudio(isOn: true);
room.isRoomJoined = true;
emit(UpdateRoomUI());
} catch (e) {
print(e);
}
});
on<OnPeerUpdateEvent>((event, emit) async {
room.remotePeers = await SdkInitializer.hmssdk.getPeers();
if (event.update == HMSPeerUpdate.peerJoined) {
if (event.peer.role.name == 'host') {
await roomsCollection?.doc(room.roomId).update({
'speakers': FieldValue.arrayUnion([event.peer.peerId])
});
}
} else if (event.update == HMSPeerUpdate.peerLeft) {
room.requests.removeWhere((req) => req.peerId == event.peer.peerId);
final requests = room.requests.map((req) => req.toJson()).toList();
await roomsCollection?.doc(room.roomId).update({'requests': requests});
}
emit(UpdateRoomUI());
});
End Room
on<EndRoomEvent>((event, emit) async {
room.isRoomJoined = false;
room.isHandRaised = false;
room.chats = [];
if (room.time != null) {
await roomsCollection?.doc(room.roomId).delete();
}
SdkInitializer.hmssdk.leave();
emit(EndRoomState(host: event.host));
});
on<EndRoomByRemoteHostEvent>((event, emit) {
room.isRoomJoined = false;
room.chats = [];
this.add(EndRoomEvent(host: true));
emit(EndRoomByRemoteHostEventState());
});
For sending the request to the host we are using sendDirectMessage() method provided by hmssdk as show in the code below:
void sendDirectMessage({
required String message,
required HMSPeer peerTo,
String? type,
HMSActionResultListener? hmsActionResultListener,
})
on<SpeakerRequestEvent>((event, emit) {
if (room.localPeer != null) {
int hostIndex = (room.remotePeers ?? []).indexWhere((e) => e.role.name == 'host');
if (hostIndex != -1) {
HMSPeer host = room.remotePeers!.elementAt(hostIndex);
final message = {
"peer_id": room.localPeer?.peerId,
"name": room.localPeer?.name,
};
SdkInitializer.hmssdk.sendDirectMessage(
message: json.encode(message),
type: room.isHandRaised ? "remove-request" : "request",
peerTo: host,
);
room.isHandRaised = !room.isHandRaised;
emit(UpdateRoomUI());
}
}
});
The host can see the list of all the peers who have raised their hands to become a speaker in an ongoing space. The host has the ability to accept or reject the request.
When the host accepts the request of any peer _approveSpeakerRequest() method is invoked. For sending the approved request to the peer we are using sendBroadcastMessage() method provided by hmssdk as show in the code below:
void _approveSpeakerRequest(BuildContext context, SpeakerRequestModel? speaker) async {
final message = {
"peer_id": speaker!.peerId,
"name": speaker.name,
};
SdkInitializer.hmssdk.sendBroadcastMessage(
message: json.encode(message),
type: "approve",
);
}
Mute/ Unmute Audio
To mute or unmute the mic feature use the below code:
if (room.localPeer?.role.name == 'host' || isSpeaker)
ResponsiveRowColumnItem(
child: GestureDetector(
onTap: (() {
room.isMicOn = !room.isMicOn;
setState(() {});
if (room.isMicOn) {
SdkInitializer.hmssdk.switchAudio();
} else {
SdkInitializer.hmssdk.switchAudio(isOn: true);
}
}),
child: Container(
height: 38,
width: 38,
child: room.isMicOn
? SvgPicture.asset(RoomsImagesAssets.kUnMuteIcon)
: SvgPicture.asset(
RoomsImagesAssets.kMuteIcon,
),
),
),
),
Chat Feature
The final feature we would be implementing is the Chat feature, which enables users to send messages to everyone present in the room, as well as get a reply from anyone. 100ms supports chat feature for every video/audio room you create.
100ms provides three ways of addressing messages :
Broadcast messages are sent to Everyone in the chat ( hmsSDK.sendBroadcastMessage).
Direct messages are sent to a specific person( hmsSDK.sendDirectMessage).
Group messages are sent to everyone with a particular HMSRole (hmsSDK.sendGroupMessage).
For more detailed information on implementing chat feature refer to the following link: docs
We are using sendBroadcastMessage() method for implementing chat feature for all the participants in the room.
Expanded(
flex: 1,
child: TextField(
style: TextStyle(color: kGreyShade5),
controller: _chatController,
decoration: InputDecoration(
contentPadding: const EdgeInsets.symmetric(horizontal: 12, vertical: 10),
focusedBorder: OutlineInputBorder(
borderRadius: BorderRadius.circular(30),
borderSide: const BorderSide(
color: kGreyShade,
width: 2,
),
),
enabledBorder: OutlineInputBorder(
borderRadius: BorderRadius.circular(30),
borderSide: const BorderSide(color: kGreyShade, width: 2),
),
suffixIcon: IconButton(
onPressed: () async {
if (_chatController.text.isNotEmpty) {
final user = FirebaseAuth.instance.currentUser;
final message = {
'username': user?.displayName,
'photo_url': user?.photoURL,
'message': _chatController.text
};
if (addChat != null)
addChat(
MessageDetails(
message: _chatController.text,
username: user?.displayName,
photoURL: user?.photoURL,
),
);
_chatController.text = '';
SdkInitializer.hmssdk.sendBroadcastMessage(
message: json.encode(message),
type: 'chat',
);
}
},
icon: Icon(
Icons.send,
color: kWhite,
)),
hintText: kHintSaying,
hintStyle: const TextStyle(color: kGreyShade, fontSize: 12),
border: InputBorder.none),
))
if (event.message.type == 'chat') {
final MessageDetails messageDetails = MessageDetails.fromJson(json.decode(event.message.message));
room.chats.add(messageDetails);
emit(UpdateRoomUI());
return;
Report a Speaker
The users can also report a speaker by clicking on the Report Speaker button on the room details screen. The data of the reported speaker will be stored in our backend (Firestore database).
import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:firebase_auth/firebase_auth.dart';
class ReportRepository {
final FirebaseFirestore _firestore = FirebaseFirestore.instance;
Future<void> createSpeakerReport(String reporterUid, String speakerUid, String description, String reason) async {
final User? user = FirebaseAuth.instance.currentUser;
try {
// store data in firestore database
final CollectionReference report = _firestore.collection('report');
await report.doc(user?.uid).set({
"reporterUid": user?.uid,
"speakerUid": speakerUid,
"description": description,
"reason": reason,
});
} catch (e) {
return null;
}
}
}
Conclusion
Congratulations for making it till the end of the article. In this article, you learned about 100ms and how you can easily integrate 100ms to build an audio room application such as "SpeekUp". However, this is only the beginning, you can learn more about changing roles, changing tracks, adding peers, direct chat with another peer, and much more from 100 ms official docs .
My overall experience working with 100ms SDK in flutter was really great as it allows us to add live audio/video, chat and many more features in our app with very less lines of code and handles all the functionality in the SDK. It supports Web, Android and iOS platforms which makes it more convincing to use.
I would recommend you guys check it out and build your own live apps!